Merging databases

Many companies are conducting a number of continuous market research surveys, which each have specific objectives. Some have a long history or have a large number of respondents.

Each of these surveys is managed by a different supplier, has different questions and database structure.

It is possible to link/merge these different databases, and

  • combine conclusions to come to more actionable insights that can not achieve by looking at the individual surveys.
  • find relationships between trends of observations across different surveys (in different countries/year).
  • explain trends seen in one survey, with data that comes from other surveys.
  • predict what may happen to the customer base, based on combining these surveys. (Predictive modeling).
  • The different surveys are read (on respondent level) and stored into a NoSQL database optimized for search and fast aggregation.
  • The solution we work with is Elastic Search (see further). We have been working with Elastic Search intensively over the last two and a half years.
  • Elasticsearch provides analysts with immediate access to results and analyses, as well as the opportunity to combine any number of demographic breaks and filters to segment large data sets.
  • The results of the query will be visualized for the analyst on a chart (e.g. line charts for plotting trends) and can be exported (e.g. to excel).
  • An analytical module will examine the statistical dependency between the variables on aggregated results.
  • With this solution, it will also be possible to get more insight from the longitudinal surveys you are conducting.
  • Elasticsearch will enable you to analyze surveys in a big data environment and will provide opportunities to detect new patterns and anomalies – and anomalies drive innovation.

Projects can be broad timelines, enabling you to learn from past mistakes, know your customer better and to uncover future opportunities. These broad timelines can be built by connecting surveys to a single project and continuously collect, scrape, store, combine and enrich data on your customers. Being able to draw insights from the never-ending stream of disparate data points is essential.

True value is on the intersection between transactional data (what people do) and opinionated data (why they are doing it). It’s important to connect what your customers are saying to their behaviour, as well as to take the context into account.

New insights are gained when structured and unstructured data are analysed together.

Advanced statistical tools and algorithms help us to generate predictive insights which then can be summarise in an easy and accessible way so clients can act upon.

Increased storage capacity demands increased sophistication in the analysis and use of that data. Making data tell a story isn’t just a matter of presenting results; it involves making connections, then going back to verify them.