Sunday, April 18, 2021

System design: Elasticsearch | wiki article

 

Features[edit]

Elasticsearch can be used to search all kinds of documents. It provides scalable search, has near real-time search, and supports multitenancy.[5] "Elasticsearch is distributed, which means that indices can be divided into shards and each shard can have zero or more replicas. Each node hosts one or more shards, and acts as a coordinator to delegate operations to the correct shard(s). Rebalancing and routing are done automatically".[5] Related data is often stored in the same index, which consists of one or more primary shards, and zero or more replica shards. Once an index has been created, the number of primary shards cannot be changed.[22]

Elasticsearch is developed alongside a data collection and log-parsing engine called Logstash, an analytics and visualisation platform called Kibana, and Beats, a collection of lightweight data shippers. The four products are designed for use as an integrated solution, referred to as the "Elastic Stack" (formerly the "ELK stack").[23]

Elasticsearch uses Lucene and tries to make all its features available through the JSON and Java API. It supports facetting and percolating,[24] [25] which can be useful for notifying if new documents match for registered queries. Another feature is called "gateway" and handles the long-term persistence of the index;[26] for example, an index can be recovered from the gateway in the event of a server crash. Elasticsearch supports real-time GET requests, which makes it suitable as a NoSQL datastore,[27] but it lacks distributed transactions.[28]

On 20 May 2019, Elastic made the core security features of the Elastic Stack available free of charge, including TLS for encrypted communications, file and native realm for creating and managing users, and role-based access control for controlling user access to cluster APIs and indexes.[29] The corresponding source code is available under the “Elastic License”, a source-available license.[30] In addition, Elasticsearch now offers SIEM [31] and Machine Learning [32] as part of its offered services.

Managed services[edit]

Developed from the Found acquisition by Elastic in 2015,[33] Elastic Cloud is a family of Elasticsearch-powered SaaS offerings which include the Elasticsearch Service, as well as Elastic App Search Service, and Elastic Site Search Service which were developed from Elastic's acquisition of Swiftype.[34] In late 2017, Elastic formed partnerships with Google to offer Elastic Cloud in GCP, and Alibaba to offer Elasticsearch and Kibana in Alibaba Cloud.

Elasticsearch Service on Elastic Cloud is the official hosted and managed Elasticsearch and Kibana offering from the creators of the project since August 2018[35][36] Elasticsearch Service users can create secure deployments with partners, Google Cloud Platform (GCP)  and Alibaba Cloud.[37][38]

AWS offers Elasticsearch as a managed service since 2015.[39][40][41] Such managed services provide hosting, deployment, backup and other support.[42] Most managed services also include support for Kibana.[citation needed]

Elasticsearch is the basis of Pangeanic's contribution to the EU's Marie Curie research project "EXPERT"[43] called ActivaTM. Pangeanic built a bilingual database compatible with Computer-Assisted Translation tools, which could offer real-time access via API from a variety of tools. The project received further funding from the EU as the National and European Central Translation Memory project[44] under the Connecting Europe Facility (CEF) programme. NEC TM aims to centralise national translation assets in all the EU's Member States so countries can re-use bilingual translation data produced as a result of public procurement contracts.

See also[edit]

References[edit]

  1. ^ "Elasticsearch Releases". Retrieved 2 December 2020 – via GitHub.
  2. ^ Kearns, Steve. "Elastic 7.11 released: General availability of searchable snapshots and the new cold tier, and the beta of schema on read". Elastic Blog. Retrieved 12 February 2021.
  3. ^ Jump up to:a b "'It's not OK': Elastic takes aim at AWS, at the risk of major collateral damage". Protocol — The people, power and politics of tech. 21 January 2021. Retrieved 22 January 2021.
  4. ^ "No, Elastic X-Pack is not going to be open source - according to Elastic themselves -". Flax.co.uk. 2 March 2018. Retrieved 14 March 2019.
  5. ^ Jump up to:a b c "Official Website". Elasticsearch.org. Retrieved 4 February 2014.
  6. ^ "DB-Engines Ranking - popularity ranking of search engines". db-engines.com. Retrieved 10 January 2016.
  7. ^ Jump up to:a b c Banon, Shay. "The Future of Compass & ElasticSearch".
  8. ^ Banon, Shay (8 February 2010). "You Know, for Search". Archived from the original on 16 January 2013.
  9. ^ "Immediate Insight from Data Matters". elastic.co. Retrieved 25 March 2015.
  10. ^ "ElasticSearch Scores $70M In Series C To Fund Growth Spurt". TechCrunch. AOL. Retrieved 25 March 2015.
  11. ^ "Elasticsearch Changes Name to Elastic to Reflect Wide Adoption Beyond Search". Elastic.co. Retrieved 19 October2016.

  12. Features[edit]

    Elasticsearch can be used to search all kinds of documents. It provides scalable search, has near real-time search, and supports multitenancy.[5] "Elasticsearch is distributed, which means that indices can be divided into shards and each shard can have zero or more replicas. Each node hosts one or more shards, and acts as a coordinator to delegate operations to the correct shard(s). Rebalancing and routing are done automatically".[5] Related data is often stored in the same index, which consists of one or more primary shards, and zero or more replica shards. Once an index has been created, the number of primary shards cannot be changed.[22]

    Elasticsearch is developed alongside a data collection and log-parsing engine called Logstash, an analytics and visualisation platform called Kibana, and Beats, a collection of lightweight data shippers. The four products are designed for use as an integrated solution, referred to as the "Elastic Stack" (formerly the "ELK stack").[23]

    Elasticsearch uses Lucene and tries to make all its features available through the JSON and Java API. It supports facetting and percolating,[24] [25] which can be useful for notifying if new documents match for registered queries. Another feature is called "gateway" and handles the long-term persistence of the index;[26] for example, an index can be recovered from the gateway in the event of a server crash. Elasticsearch supports real-time GET requests, which makes it suitable as a NoSQL datastore,[27] but it lacks distributed transactions.[28]

    On 20 May 2019, Elastic made the core security features of the Elastic Stack available free of charge, including TLS for encrypted communications, file and native realm for creating and managing users, and role-based access control for controlling user access to cluster APIs and indexes.[29] The corresponding source code is available under the “Elastic License”, a source-available license.[30] In addition, Elasticsearch now offers SIEM [31] and Machine Learning [32] as part of its offered services.

    Managed services[edit]

    Developed from the Found acquisition by Elastic in 2015,[33] Elastic Cloud is a family of Elasticsearch-powered SaaS offerings which include the Elasticsearch Service, as well as Elastic App Search Service, and Elastic Site Search Service which were developed from Elastic's acquisition of Swiftype.[34] In late 2017, Elastic formed partnerships with Google to offer Elastic Cloud in GCP, and Alibaba to offer Elasticsearch and Kibana in Alibaba Cloud.

    Elasticsearch Service on Elastic Cloud is the official hosted and managed Elasticsearch and Kibana offering from the creators of the project since August 2018[35][36] Elasticsearch Service users can create secure deployments with partners, Google Cloud Platform (GCP)  and Alibaba Cloud.[37][38]

    AWS offers Elasticsearch as a managed service since 2015.[39][40][41] Such managed services provide hosting, deployment, backup and other support.[42] Most managed services also include support for Kibana.[citation needed]

    Elasticsearch is the basis of Pangeanic's contribution to the EU's Marie Curie research project "EXPERT"[43] called ActivaTM. Pangeanic built a bilingual database compatible with Computer-Assisted Translation tools, which could offer real-time access via API from a variety of tools. The project received further funding from the EU as the National and European Central Translation Memory project[44] under the Connecting Europe Facility (CEF) programme. NEC TM aims to centralise national translation assets in all the EU's Member States so countries can re-use bilingual translation data produced as a result of public procurement contracts.

    See also[edit]

    References[edit]

    1. ^ "Elasticsearch Releases". Retrieved 2 December 2020 – via GitHub.
    2. ^ Kearns, Steve. "Elastic 7.11 released: General availability of searchable snapshots and the new cold tier, and the beta of schema on read". Elastic Blog. Retrieved 12 February 2021.
    3. ^ Jump up to:a b "'It's not OK': Elastic takes aim at AWS, at the risk of major collateral damage". Protocol — The people, power and politics of tech. 21 January 2021. Retrieved 22 January 2021.
    4. ^ "No, Elastic X-Pack is not going to be open source - according to Elastic themselves -". Flax.co.uk. 2 March 2018. Retrieved 14 March 2019.
    5. ^ Jump up to:a b c "Official Website". Elasticsearch.org. Retrieved 4 February 2014.
    6. ^ "DB-Engines Ranking - popularity ranking of search engines". db-engines.com. Retrieved 10 January 2016.
    7. ^ Jump up to:a b c Banon, Shay. "The Future of Compass & ElasticSearch".
    8. ^ Banon, Shay (8 February 2010). "You Know, for Search". Archived from the original on 16 January 2013.
    9. ^ "Immediate Insight from Data Matters". elastic.co. Retrieved 25 March 2015.
    10. ^ "ElasticSearch Scores $70M In Series C To Fund Growth Spurt". TechCrunch. AOL. Retrieved 25 March 2015.
    11. ^ "Elasticsearch Changes Name to Elastic to Reflect Wide Adoption Beyond Search". Elastic.co. Retrieved 19 October2016.
    12. ^ Schleifer, Theodore (21 June 2018). "The IPOs keep coming: The search company Elastic has filed to go public". Recode. Archived from the original on 5 October 2018. Retrieved 22 June 2018.
    13. ^ Banon, Shay (5 October 2018). "Ze Bell Has Rung: Thank You Users, Customers, and Partners". Elastic (NV). Retrieved 24 October 2018.
    14. ^ Banon, Shay. "Doubling down on open, Part II". Elastic. Retrieved 19 January 2021.
    15. ^ Jump up to:a b Vaughan-Nichols, Steven J. "Elastic changes open-source license to monetize cloud-service use". ZDNet. Retrieved 23 January 2021.
    16. ^ Banon, Shay. "Amazon: NOT OK - why we had to change Elastic licensing". Elastic. Retrieved 19 January 2021.
    17. ^ "Stepping up for a truly open source Elasticsearch". Amazon Web Services. 21 January 2021. Retrieved 28 January2021.
    18. ^ Vaughan-Nichols, Steven J. "AWS, as predicted, is forking Elasticsearch". ZDNet. Retrieved 28 January 2021.
    19. ^ "CrateDB Doubling Down on Permissive Licensing and the Elasticsearch Lockdown". CrateDB. 27 January 2021. Retrieved 28 January 2021.
    20. ^ "Momentum Builds to Break Elasticsearch Licensing Deadlock". Datanami. 25 January 2021. Retrieved 31 January 2021.
    21. ^ Anderson, Tim (13 April 2021). "You know what? Fork this: AWS renames its take on Elasticsearch to OpenSearch following trademark fight". The Register. Retrieved 13 April2021.
    22. ^ "How to monitor Elasticsearch performance".
    23. ^ "Elastic brings order to its product line with Elastic Stack". Social.techcrunch.com. Retrieved 1 April 2019.
    24. ^ "percolate at elasticsearch.org reference". Elasticsearch.org. Archived from the original on 2 October 2013. Retrieved 4 February 2014.
    25. ^ "Percolating" is a term peculiar to Elasticsearch. Percolating is a reverse search: instead of returning all the documents that match a search query, percolating returns all the (stored) search queries that match a document as their output. Nunn, Xavier; "Detecting data leaks in real time with a custom percolator", Serena Capital blogs, 2019-January-8
    26. ^ "elasticsearch Guide: Gateway". Elasticsearch.org. Retrieved 19 April 2013.
    27. ^ "Elasticsearch as database". Karussell.wordpress.com. Retrieved 4 February 2014.
    28. ^ "No transaction support". Elasticsearch-users.115913.n3.nabble.com. 8 July 2010. Retrieved 4 February 2014.
    29. ^ "Security for Elasticsearch is now free". Elastic Blog. 20 May 2019. Retrieved 17 June 2019.
    ^     Schleifer, Theodore (21 June 2018). "The IPOs keep coming: The search company Elastic has filed to go public". Recode. Archived from the original on 5 October 2018. Retrieved 22 June 2018.
  13. ^ Banon, Shay (5 October 2018). "Ze Bell Has Rung: Thank You Users, Customers, and Partners". Elastic (NV). Retrieved 24 October 2018.
  14. ^ Banon, Shay. "Doubling down on open, Part II". Elastic. Retrieved 19 January 2021.
  15. ^ Jump up to:a b Vaughan-Nichols, Steven J. "Elastic changes open-source license to monetize cloud-service use". ZDNet. Retrieved 23 January 2021.
  16. ^ Banon, Shay. "Amazon: NOT OK - why we had to change Elastic licensing". Elastic. Retrieved 19 January 2021.
  17. ^ "Stepping up for a truly open source Elasticsearch". Amazon Web Services. 21 January 2021. Retrieved 28 January2021.
  18. ^ Vaughan-Nichols, Steven J. "AWS, as predicted, is forking Elasticsearch". ZDNet. Retrieved 28 January 2021.
  19. ^ "CrateDB Doubling Down on Permissive Licensing and the Elasticsearch Lockdown". CrateDB. 27 January 2021. Retrieved 28 January 2021.
  20. ^ "Momentum Builds to Break Elasticsearch Licensing Deadlock". Datanami. 25 January 2021. Retrieved 31 January 2021.
  21. ^ Anderson, Tim (13 April 2021). "You know what? Fork this: AWS renames its take on Elasticsearch to OpenSearch following trademark fight". The Register. Retrieved 13 April2021.
  22. ^ "How to monitor Elasticsearch performance".
  23. ^ "Elastic brings order to its product line with Elastic Stack". Social.techcrunch.com. Retrieved 1 April 2019.
  24. ^ "percolate at elasticsearch.org reference". Elasticsearch.org. Archived from the original on 2 October 2013. Retrieved 4 February 2014.
  25. ^ "Percolating" is a term peculiar to Elasticsearch. Percolating is a reverse search: instead of returning all the documents that match a search query, percolating returns all the (stored) search queries that match a document as their output. Nunn, Xavier; "Detecting data leaks in real time with a custom percolator", Serena Capital blogs, 2019-January-8
  26. ^ "elasticsearch Guide: Gateway". Elasticsearch.org. Retrieved 19 April 2013.
  27. ^ "Elasticsearch as database". Karussell.wordpress.com. Retrieved 4 February 2014.
  28. ^ "No transaction support". Elasticsearch-users.115913.n3.nabble.com. 8 July 2010. Retrieved 4 February 2014.
  29. ^ "Security for Elasticsearch is now free". Elastic Blog. 20 May 2019. Retrieved 17 June 2019.
Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)