Before you add Elasticsearch, measure the MySQL search you already have

Adding Elasticsearch to a React, GraphQL, and MySQL stack is usually justified with three words: search is slow. Then the team spends a quarter standing up a cluster, a sync pipeline, and a reindex job, ships it, and never writes down how slow MySQL actually was. So nobody can say whether it worked. The new search feels faster, and feels-faster is not a number.

The only Elasticsearch test that matters is the one against the MySQL baseline you measured first. Everything else is detail.

That reframes the whole QA effort. You are not testing a search engine. You are testing a bet: that Elasticsearch is enough faster, and enough better, to justify a second system you now have to keep in sync forever. Test the bet, behind a flag you can flip back, and the strategy writes itself.

Measure the thing you're replacing, first

Before any Elasticsearch code lands, capture the current MySQL search latency under realistic data volume and concurrency. Not a single query on an empty dev table. The p95 and p99 at production scale, which is the number users actually feel.

// Capture a distribution, not one lucky query.
async function baseline(term, runs = 200) {
  const times = [];
  for (let i = 0; i < runs; i++) {
    const t = performance.now();
    await graphqlClient.query({ query: SEARCH_PRODUCTS, variables: { term } });
    times.push(performance.now() - t);
  }
  times.sort((a, b) => a - b);
  return { p50: times[runs * 0.5 | 0], p95: times[runs * 0.95 | 0], p99: times[runs * 0.99 | 0] };
}

Write that number down somewhere the whole team can see it. It is the bar. If Elasticsearch does not clear it by a margin that justifies the operational cost, the honest answer is to not ship it, and you can only give that answer if you measured.

Put it behind a flag that falls back to MySQL

The single most important piece of test infrastructure is not a test. It is the flag that lets you turn Elasticsearch off in production without a deploy. Build the resolver to fall back to MySQL the moment Elasticsearch errors or the flag flips, and you have made the scary part of this migration reversible.

async function searchProducts(_, { query, filters }) {
  if (flags.isEnabled('use_elasticsearch')) {
    try {
      return await es.search(query, filters);
    } catch (err) {
      logger.error('ES search failed, falling back to MySQL', err);
      return mysql.search(query, filters); // degrade, do not 500
    }
  }
  return mysql.search(query, filters);
}

Now your rollout is a dial, not a leap: 5% of traffic, then 25, then 100, watching the latency delta against the baseline at each step, with a one-flag rollback if the number turns bad. The QA test here is simple and worth automating: flip the flag off and assert search still returns correct results from MySQL.

The expensive failure is silent drift, not bad relevance

This is the part I got wrong early. The instinct is to write a wall of relevance tests first: exact match, fuzzy match, filters, pagination. Those matter, but they are not what pages you at 3am. What pages you is drift: a write that updated MySQL and silently never made it into Elasticsearch, so search is confidently returning stale or missing data and every relevance test still passes because they run against a freshly indexed fixture.

So the test that earns its keep is a consistency check that compares the two systems on real records and runs on a schedule, not just in CI.

async function findDrift(sampleSize = 1000) {
  const rows = await mysql.query(
    'SELECT id, name, price, updated_at FROM products ORDER BY updated_at DESC LIMIT ?', [sampleSize]
  );
  const drift = [];
  for (const r of rows) {
    const doc = await es.get({ index: 'products', id: r.id }).catch(() => null);
    if (!doc) { drift.push({ id: r.id, issue: 'missing in ES' }); continue; }
    if (doc._source.price !== r.price) {
      drift.push({ id: r.id, issue: 'price mismatch', mysql: r.price, es: doc._source.price });
    }
  }
  return drift;
}

Sample the most recently updated rows, because that is where sync lag shows up first. Alert when drift crosses a threshold. This one check catches the class of bug that the entire relevance suite will cheerfully miss.

What to actually test

Strip the plan down to the handful of things that defend the bet:

1. The delta: Elasticsearch p95 versus the MySQL baseline, on the same queries, at the same scale. This is the headline.
2. Drift: the scheduled MySQL-versus-ES consistency check above, alerting on lag.
3. The fallback: flip the flag off and on, assert search keeps working and returns correct results both ways.
4. Sync on writes: create, update, and delete through your real GraphQL mutations, then assert the change appears (or disappears) in Elasticsearch within your sync window.

That is the suite that matters. Notice what is not on it: a generic "test every search permutation" matrix, chaos-killing cluster nodes you do not run in prod, and load tests for traffic you do not have. Add those when you actually have the scale to justify them, not because a checklist said to.

So: one bet, measured, reversible

Elasticsearch is worth adding when it clears a baseline you measured, stays in sync with the source of truth you can prove, and can be turned off with a flag instead of a deploy. Get those three and the migration is boring, which is the goal. Skip the baseline and you are not running a QA strategy, you are running on a feeling that the new thing is faster, and you will not find out you were wrong until a user does.