[BUG] Session crashes when using text search and detection labels together

[frpunzalan]

Describe the problem

When applying compute_similarity to a dataset with detection labels (ground truth data in this case), the session crashes when the detection labels are inspected after doing a text search.

Code to reproduce issue

1. Install fiftyone (v0.25.0) through poetry

# this assumes poetry is installed in the system
$ poetry add fiftyone==0.25.0
# install dependencies
$ poetry add open-clip-torch torchvision

2. Run the code described below

$ poetry run python tmp.py
...
where `[<username>@]<hostname>` refers to your current machine. The App can
then be viewed in your browser at http://localhost:5151.
...

3. Open the app from browser: http://localhost:5151
4. Apply a text search
5. Open the labels in the left hand side.

System information

• OS Platform and Distribution: Linux Ubuntu 24.04
• Python version: Python 3.12.3
• FiftyOne version: FiftyOne v0.25.0, Voxel51, Inc.
• FiftyOne installed from: poetry

Other info/logs

The following shows the error stack trace.

stack trace

File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/graphql/execution/execute.py", line 1036, in await_result
    return build_response(await result, errors)  # type: ignore
                          ^^^^^^^^^^^^
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/graphql/execution/execute.py", line 451, in get_results
    await gather(*(results[field] for field in awaitable_fields)),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/graphql/execution/execute.py", line 535, in await_result
    self.handle_field_error(error, return_type)
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/graphql/execution/execute.py", line 569, in handle_field_error
    raise error
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/graphql/execution/execute.py", line 528, in await_result
    return_type, field_nodes, info, path, await result
                                          ^^^^^^^^^^^^
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/strawberry/schema/schema_converter.py", line 513, in _async_resolver
    return await await_maybe(_get_result(_source, strawberry_info, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/strawberry/utils/await_maybe.py", line 12, in await_maybe
    return await value
           ^^^^^^^^^^^
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/fiftyone/server/aggregations.py", line 120, in aggregate_resolver
    view = await _load_view(form, form.slices)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/fiftyone/server/aggregations.py", line 196, in _load_view
    return await fosv.get_view(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/fiftyone/core/utils.py", line 2343, in run_sync_task
    return await loop.run_in_executor(_get_sync_task_executor(), func, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/fiftyone/server/view.py", line 133, in run
    view = get_extended_view(
           ^^^^^^^^^^^^^^^^^^
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/fiftyone/server/view.py", line 172, in get_extended_view
    view = extend_view(view, extended_stages)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/fiftyone/server/view.py", line 218, in extend_view
    view = view.add_stage(stage)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/fiftyone/core/collections.py", line 4036, in add_stage
    return self._add_view_stage(stage)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/fiftyone/core/view.py", line 1712, in _add_view_stage
    stage.validate(self)
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/fiftyone/core/stages.py", line 7224, in validate
    pipeline = self._make_pipeline(sample_collection)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/fiftyone/core/stages.py", line 7246, in _make_pipeline
    return results.sort_by_similarity(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/fiftyone/brain/similarity.py", line 678, in sort_by_similarity
    query=self._parse_query(query),
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/fiftyone/brain/similarity.py", line 786, in _parse_query
    with model:
  File "/home/frpunzalan/.cache/pypoetry/virtualenvs/fiftyone-server-E5YjJ5A3-py3.12/lib/python3.12/site-packages/fiftyone/utils/torch.py", line 576, in __exit__
    self._no_grad.__exit__(*args)
    ^^^^^^^^^^^^^^^^^^^^^^

Code for recreating the crash

python code (tmp.py)

import fiftyone as fo
import fiftyone.brain as fob
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")
dataset.add_dynamic_sample_fields()

field = dataset.get_field("ground_truth")
field.description = "Ground truth annotations"
field.info = {"url": "https://fiftyone.ai"}
field.save()  # must save after edits

dataset.reload()

model_kwargs = {
    "clip_model": "ViT-B-32-quickgelu",
    "pretrained": "metaclip_400m",
    "text_prompt": "A photo of a",
}

fob.compute_similarity(
    dataset,
    model="open-clip-torch",
    model_kwargs=model_kwargs,
    brain_key="sim_metaclip",
)

session = fo.launch_app(dataset, remote=True, address="0.0.0.0", port=5151)

# Blocks execution until the App is closed
session.wait()

Willingness to contribute

• Yes. I can contribute a fix for this bug independently
• Yes. I would be willing to contribute a fix for this bug with guidance
  from the FiftyOne community
• No. I cannot contribute a bug fix at this time