annotate_score does not work on SQLite FTS backend

[gasman]

Issue Summary

Applying .annotate_score("score") to search results fails with an invalid query when the SQLite FTS backend is in use.

Steps to Reproduce

From the ./manage.py shell prompt on a bakerydemo instance:

from wagtail.models import Page
Page.objects.search("yeast").annotate_score("score")

fails with the exception

Traceback (most recent call last):
  File "/Users/matthew/.virtualenvs/wagtail/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/matthew/.virtualenvs/wagtail/lib/python3.11/site-packages/django/db/backends/sqlite3/base.py", line 357, in execute
    return Database.Cursor.execute(self, query, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: no such column: wagtailsearch_indexentry_fts

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/Users/matthew/Development/tbx/wagtail/devscript/wagtail/wagtail/search/backends/base.py", line 308, in __repr__
    data = list(self[:21])
           ^^^^^^^^^^^^^^^
  File "/Users/matthew/Development/tbx/wagtail/devscript/wagtail/wagtail/search/backends/base.py", line 302, in __iter__
    return iter(self.results())
                ^^^^^^^^^^^^^^
  File "/Users/matthew/Development/tbx/wagtail/devscript/wagtail/wagtail/search/backends/base.py", line 268, in results
    self._results_cache = list(self._do_search())
                               ^^^^^^^^^^^^^^^^^
  File "/Users/matthew/Development/tbx/wagtail/devscript/wagtail/wagtail/search/backends/database/sqlite/sqlite.py", line 624, in _do_search
    return list(self.get_queryset())
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/matthew/.virtualenvs/wagtail/lib/python3.11/site-packages/django/db/models/query.py", line 394, in __iter__
    self._fetch_all()
  File "/Users/matthew/.virtualenvs/wagtail/lib/python3.11/site-packages/django/db/models/query.py", line 1867, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/matthew/.virtualenvs/wagtail/lib/python3.11/site-packages/django/db/models/query.py", line 87, in __iter__
    results = compiler.execute_sql(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/matthew/.virtualenvs/wagtail/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1398, in execute_sql
    cursor.execute(sql, params)
  File "/Users/matthew/.virtualenvs/wagtail/lib/python3.11/site-packages/django/db/backends/utils.py", line 103, in execute
    return super().execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/matthew/.virtualenvs/wagtail/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/matthew/.virtualenvs/wagtail/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/matthew/.virtualenvs/wagtail/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/Users/matthew/.virtualenvs/wagtail/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/Users/matthew/.virtualenvs/wagtail/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/matthew/.virtualenvs/wagtail/lib/python3.11/site-packages/django/db/backends/sqlite3/base.py", line 357, in execute
    return Database.Cursor.execute(self, query, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.OperationalError: no such column: wagtailsearch_indexentry_fts

Incidentally, applying annotate_score("score") to a Page.objects.search(MATCH_ALL) query (as this test does) does return a result set, but the score attribute is still missing from the objects.

• I have confirmed that this issue can be reproduced as described on a fresh Wagtail project: yes

Technical details

• Python version: 3.11.1
• Django version: 4.1.6
• Wagtail version: 5.0a0 (main)

[dopry]

I can confirm this issue as well. I had to implement a work around for it when implementing a searchScore field for the grapple interface and also applies to wagtail 4.1 and 4.2.

[gasman]

Page.objects.search(MATCH_ALL).annotate_score("score") also fails to annotate the queryset on the Postgres backend, presumably because it's bypassing running an actual search query.