Pagination of results in Elasticsearch | from & size, scroll, search after [ES7 for Beginners #4.4]

In this elastic search tutorial, we discuss about Paginating the search results or search result Pagination. This is part of Query DSL (Domain Specific Language)

Sections:
Simple Pagination using from and size (0:00)
Scaling problem with from and size pagination (3:00)
Elasticsearch Pagination Scroll Request (5:05)
Scaling problem with scroll pagination request (9:00)
Search after pagination in Elasticsearch (9:55)
Things to note about Elasticsearch search after pagination API (12:32)

Hashtags:
#coding #theory #computerscience #elasticsearch #clusters #distributedSystems #tutorial #logstash #kibana #beats #aws #dataScience #pagination #queryDSL

Places where these ideas can be used:
- To paginate the search results using ElasticSearch
- To build infinite scroll using ElasticSearch

Some Theory:

Elasticsearch is a search engine based on the Lucene library. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents.

To paginate through a larger set of results, you can use the search API’s size and from parameters. The size parameter is the number of matching documents to return. The from parameter is a zero-indexed offset from the beginning of the complete result set that indicates the document you want to start with.

While a search request returns a single “page” of results, the scroll API can be used to retrieve large numbers of results (or even all results) from a single search request, in much the same way as you would use a cursor on a traditional database.

(This tutorial is part of a series of tutorials on Elasticsearch, logstash and Kibana. It uses docker for purpose of installation, and may even use aws in the future.)

[m.t]

After watching this video I have understood the pagination concept very clearly.Thanks Alot For posting this video.

[poulamibag]

Thank you for this. Great explanation @Codetuber! I wish more people come to your channel and benefit themselves. Keep up the good work.

[lakshmigopalan]

Your video on pagination is really great.. thanks for explaining so clearly

[tigerace]

this is awesome material dude!! i´ve been watching your vid´s cause i need to aprobe my elasticsearch cert´ exam next week u_u (i´m kinda scared), please wish me look bro!!

[sujoydey]

Thank you for the great explanation. Could you also please explain about the Point in time (PIT) api and how and why it should be used with search_after. Also how is point in time different from search context discussed in scroll api

[gauson]

Could you please make a video on point of time api and how to use it for pagination. The data is getting refreshed in the background and we are missing some of the data. It will be of great help. Thank you

[cqyjezx]

when will the next video be released? Thank you and Congrats for the succesfull serie

[yhqimqo]

Which search api better to search files

[vikaskumar]

Its not searching beyond 10K results even when I use search_after. Is this something related to max_result_window ?

[lakshmigopalan]

i have a field which is of text datatype which has values in alphanumeric characters and i am using "terms" to search multiple values but i am not getting the results properly example : {"terms" : { "rollnum" : ["123-IN", "345"]}}. I am getting results only for 345. how should i handle this?