This article is the second part of a tutorial which describes how to build a search-as-you-type feature based on Elasticsearch, Python/Flask and AngularJS. The first part has discussed how to set-up Elasticsearch and a microservice in Python/Flask, i.e. the back-end part of the system. It also provided an overall view on the architecture. In this […]
Tag Archives: ElasticSearch
It used to be the only time you’d be looking at your application logs was when something went wrong.
Full-text search, unlike most of the topics in this machine learning series, is a problem that most web developers have encountered at some point in their daily work. A client asks you to put a search field somewhere, and you write some SQL along the lines of
WHERE title LIKE %:query%. It’s convincing at first, but then a few days later the client calls you and claims that “search is broken!”
Of course, your search isn’t broken, it’s just not doing what the client wants. Regular web users don’t really understand the concept of exact matches, so your search quality ends up being poor. You decide you need to use full-text search. With some MySQL fidgeting you’re able to set up a FULLTEXT index and use a more evolved syntax, the “MATCH() … AGAINST()” query.
Great! Problem solved. For smallish databases.
As you hit the hundreds of thousands of records, you notice that your database is sluggish. MySQL just isn’t great at full-text search. So you grab ElasticSearch, refactor your code a bit, and deploy a Lucene-driven full-text search cluster that works wonders. It’s fast and the quality of results is great.
Which leads you to ask: what the heck is Lucene doing so right?
This article (on TF-IDF, Okapi BM-25, and relevance scoring in general) and the next one (on inverted indices) describe the basic concepts behind full-text search.
Couchbase and Elasticsearch are both optimized for JSON. Where do the two technologies come together and what are the best practices in using them? This session from David Maier of Couchbase and Bala Venkatrao from Elastic will answer exactly those questions. After a quick overview and discussion of use cases, we will describe how the Elasticsearch plugin is set up and then configured to index data from Couchbase. In addition, we will also showcase the use of Kibana for data visualization on top of Elasticsearch. You will come away armed with the knowledge needed to successfully deploy these technologies together in your apps.
ElasticSearch is an open source search engine with a REST API for querying written words. Elasticsearch is a standalone database server, written in Java. It is built on top of Apache Lucene. It uses JSON over HTTP and is suitable for programming languages other than Java as well. The first version of ElasticSearch was released in February 2010 by Shay Banon. ElasticSearch exposes a REST API to interact with data using HTTP verbs. To retrieve information, we use a GET command. To create or update records, you can use a PUT or POST command.
For the complete Feature List please visit https://www.elastic.co/products/elasticsearch.
Two databases can be better as one as we show how you can use Mongoosastic to user MongoDB and Elasticsearch at the same time leveraging each one’s strengths without rebuilding your code.
At Compose we’re all about using the right database for the job, and sometimes that can mean using more than one database harnessed to one another. If that sounds strange or over complex, let’s show you an example where Elasticsearch can help overcome some of the inadequacies of MongoDB.
MongoDB is a great general purpose database but one place where its limitations show up is with its full text searching. We only got a finalised full text search mechanism in MongoDB 2.6 and even then you can only have one field per collection indexed for searching in that way. If you also want richer full text search capabilities than what MongoDB offers then you will need an alternative. Elasticsearch gives you a much more extensive and powerful search capability, being built upon the Lucene text search library, but you may not want to move your application over to another database. Apart from anything else, the other database may not be suitable for the rest of your application workload.
Dynamo Streams provides a stream of all the updates done to your DynamoDB table. It is a simple but extremely powerful primitive which will enable developers to easily build solutions like cross-region replication, and to host additional materialized views, for instance an ElasticSearch index, on top of DynamoDB tables. In this session we will dive deep into details of Dynamo Streams, and how customers can leverage Dynamo Streams to build custom solutions and to extend the functionality of DynamoDB. We will give a demo of an example application built on top of Dynamo Streams to demonstrate the power and simplicity of Dynamo Streams.