Building the future of OpenSearch together

Mon, Sep 16, 2024 · Anandhi Bumstead, Carl Meadows, Daniel Doubrovkine, Eli Fisher, Mukul Karnik

As announced today, the OpenSearch Project is now part of the newly formed OpenSearch Software Foundation, a community-driven initiative under the Linux Foundation. This marks a major milestone in the history of the OpenSearch Project, and we couldn’t be more excited to share our thoughts on what this means to the OpenSearch community.

When we launched OpenSearch in 2021, we stated that the project’s mission was “to ensure users continue to have a secure, high-quality, fully open source search and analytics suite with a rich roadmap of new and innovative functionality.” Building the structure and processes to support that mission has been a labor of love for those of us who work on the project. That very much includes our partners as well as the community of users, developers, administrators, and open-source software enthusiasts that stand behind OpenSearch with their contributions to code, documentation, product feedback, and much more. We build OpenSearch together.

We’ve worked to maintain a roadmap that prioritizes innovation across the areas that the community cares about most. Together, we’re executing against that roadmap with a collaborative development effort that has delivered 21 major and minor releases and hundreds of new and enhanced features through version 2.16.

Together, we’ve seen the project expand to 122 GitHub repositories thanks to more than a thousand contributors and the 39,000-plus merged pull requests they have made to date, not to mention the support of more than 250 maintainers across more than 25 organizations.

And together, we’ve grown OpenSearchCon from humble beginnings in a Seattle soundstage to an international series with conferences in the US, Europe, and India.

As a result of all that work, OpenSearch has been downloaded more than 700 million times.

Along the way, we’ve done a lot to help make OpenSearch a more open and welcoming project. In in May of 2021, before OpenSearch v1.0 was released, we started hosting public community meetings. A year later, we added the first maintainers not employed by AWS. We created an open process for managing security issues, including a predisclosure list, in September 2022. In April 2023 we launched a public Slack workspace to facilitate transparent discussions; the next month, we opened the release process to contributors outside of AWS. And last December we formed the OpenSearch Leadership Committee to bring together a mix of contributors from a wide range of organizations to “preserve and advance the OpenSearch Project in order to benefit the OpenSearch community and represent its views.”

We’re making progress toward the goal of a more open OpenSearch but are still very early in the journey of creating a wide and diverse base of contributors. As of this month; 42% of OpenSearch GitHub repositories have at least one non-Amazon maintainer; the percentage of contributions from non-Amazon contributors has more than doubled since 2021; and more than 80% of the 2,800-plus members of our public Slack workspace are non-Amazon employees.

While that progress is encouraging, we recognize that there is more work to be done. That work consists of opening up the project even more to encourage participation from more contributors and more organizations. The Leadership Committee recognized this as well, so as part of their charter, they focused their efforts on looking for ways to help more people get involved with OpenSearch. Ultimately, the committee’s recommendation supported the transition of OpenSearch to a vendor-neutral nonprofit organization—echoing the feedback we’ve heard from many across the OpenSearch community. In April, that transition began in earnest.

Today’s announcement brings that process to fruition as the OpenSearch Project becomes part of a vendor-neutral non-profit foundation hosted by the Linux Foundation. If you’re not already familiar with the Linux Foundation, the nonprofit organization began in 2000 and has grown to become an enabler of mass innovation through open source, with more than 100 foundations and over 980 projects under its purview. As a “foundation of foundations,” the Linux Foundation is especially well suited to take on a large, multifaceted project like the OpenSearch Project and foster its growth and innovation. The Linux Foundation focuses on leveraging best practices and addressing the needs of contributors, users, and solution providers to create sustainable models for open collaboration. Projects under the foundation receive support for project governance, trademark and legal issues, marketing, events, training, technical infrastructure, and more.

We believe this change is excellent news for everyone who contributes to, depends on, or supports open-source OpenSearch, and we hope you feel the same. Here’s what this news means to the OpenSearch community.

Open collaboration

Since OpenSearch debuted, AWS has acted as the project’s steward while prioritizing opening up important elements of the project, such as the release process, codebase maintenance, collaboration through Slack and forums, and the establishment of a Leadership Committee consisting of a majority of non-AWS members. At the same time, we recognize that there are opportunities to open up the project even further. Moving to a vendor-neutral foundation will present more opportunities for organizations to get involved with OpenSearch. For one, the Linux Foundation brings to the project well-defined, proven processes for many of the activities we’ve been working to develop on an ad hoc basis to support open collaboration. And the fact is that many organizations prioritize contributing to open-source projects that have vendor-neutral governance. We hope any organization that may have previously been hesitant about collaborating with OpenSearch considers this an open invitation to get involved.

Open governance

The OpenSearch Project will now benefit from an open governance model. A Technical Steering Committee (TSC), comprised of members who are actively contributing to the project’s technical roadmap and the advancement of the project, will guide the technical development of OpenSearch software. Separately, an elected governing board, drawn from members of the non-profit foundation and a representative from the TSC, will direct foundation funds to support the project. Each of these governing bodies will establish policies for transparency, accountability, and communication with the OpenSearch community.

Accelerating innovation

As noted above, the mission of the OpenSearch Project has always been centered on continuity and innovation. With this evolution, we expect the number of individuals and organizations contributing to the project to increase significantly. To us, the math is straightforward: more contributors representing a broader base of organizations equals an acceleration of OpenSearch’s dynamic trajectory of innovation. Many hands make light work! Moreover, with the programs and capabilities offered by the Linux Foundation, we expect accelerated progress in key areas that support contribution and innovation at scale, such as documentation, program management, operations, and more.

What won’t change

While there are a lot of benefits to this move, we want to be clear that many of the ways we work together will stay the same. The OpenSearch software will continue to be built and maintained in the same GitHub Organization, and the software will continue to be offered under version 2.0 of the open-source Apache License. This website, this blog, our forum, and our Slack instance will continue to serve as venues where you can learn about OpenSearch and connect with the OpenSearch community.

It goes without saying that our commitment to building great open-source search, analytics, observability, and vector database software, hand-in-hand with the community, is stronger than ever, and that commitment is 100% shared by the Linux Foundation. In fact, our expectation is that this move will make collaboration across the community easier than ever, helping all of us scale faster and accelerate innovation.

To everyone who has contributed to the OpenSearch Project, thank you. Your efforts have helped OpenSearch grow and achieve this remarkable milestone. This is the beginning of an incredible new chapter in the history of OpenSearch, and we’re excited to have you join us. Together, there’s so much more we can do.

To the founding members of the OpenSearch Software Foundation—including premier members AWS, SAP, and Uber, and general members Aiven, Aryn, Atlassian, Canonical, DigitalOcean, Eliatra, Graylog, NetApp Instaclustr, and Portal26—thank you for your investment in OpenSearch and your dedication to vendor-neutral, community-driven open-source software.

As always, your feedback is encouraged and appreciated. We’ve opened a forum topic for questions and discussion. For more information about the OpenSearch Software Foundation, visit https://foundation.opensearch.org.