Intelligent Search for Personalized Education

When it comes to delivering information that users are seeking, there are broadly two user experience paradigms. The first one involves a well-crafted, menu-based navigation system. The second is Search which serves content based on user queries.

Search is a far superior method by which we seek online information today. While the menu-based system iteratively leads the users to the exact information they are looking for, the limited number of menu options makes it a less feasible option, especially when the universe of information is vast. Not to mention, content discovery becomes a slow and tedious process under heaps of menus and tabs. For these reasons, it made more sense to expose Embibe’s content to users via a search-based UI.

From a product design point of view, a search-based UI is more practical as it can adapt itself to individual users. As we expand our content to thousands of exams over hundreds of syllabi and given the huge amounts of data that Embibe has already collected as users interact with the platform, it is possible to develop content discovery and surfacing techniques that are personalised to individual users, thereby solving their information need quicker and delighting them.

As stated earlier, Embibe has been collecting vast amounts of data for the past ten years, which powers our search-based personalised content discovery system. Figure 1 shows the flow of control as a user makes a query until the results are returned. Embibe’s Personalized Content Discovery is powered using in-house custom-developed pre- and post-processing frameworks over pure Elasticsearch and Solr and Apache Lucene, query rewriting, intent detection, multi-pass retrieval, result re-ranking, and query disambiguation. Details of individual components of the system and performance metrics are beyond the scope of this article.

Our search engine serves content based on user queries across the grades. It re-ranks relevant results based on the user’s grade, cohort assignments, historical search trends and content consumption patterns, and content difficulty based on the exam and past user interaction, among 25 such weighting factors. 

Question Number Template (QNT) Search: The unique feature facilitates students to directly search a question of a specific book and get the practice and solution for it. This saves a lot of time for students who are going to attend an exam in the near future. 

Additionally, students are shown recommendations for targeted content to focus their time on, based on their past interaction with the platform if sufficient data is available or based on lookalike users in data insufficient cases. 

Personalised recommendations are of two types — recommendations for academic content where the user is at a very low percentile standing compared to the rest of their performance cohort, and recommendations for question packs containing specialised cuts of behaviour-targeted practice, like mistakes of top rankers, careless mistakes of most students, etc. These practice packs help address specific user behavioural gaps.

Figure 1: Personalized Study Search Flow of Control