Describe the bug
When running the tutorial TFF simulations with accelerators (https://www.tensorflow.org/federated/tutorials/simulations_with_accelerators), the GPU utilization is almost 0.
The GPU utilization is as expected (and training runs much faster) if using the python execution context with tff.backends.native.set_local_python_execution_context().

Environment (please complete the following information):

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 22.04
• Python package versions (e.g., TensorFlow Federated, TensorFlow): TFF 0.48.0 and 0.50.0, TF 2.11.0.
• Python version: 3.10.9
• Bazel version (if building from source): N/A
• CUDA/cuDNN version: cudatoolkit 11.2.2, cudnn 8.1.0.77
• What TensorFlow Federated execution stack are you using? C++ execution context (?)

Expected behavior
The TFF simulation should use GPU and run faster.

Additional context

• The gpus are visible with tf.config.list_physical_devices('GPU').
• The same issue if present if manually setting the C++ execution context tff.backends.native.set_sync_local_cpp_execution_context().
• nvidia-smi shows that gpu memory has been allocated but utilization in training is 0%.
• The gpu is used as expected in evaluation accuracy = keras_evaluate(eval_model, test_data, metric) and nvidia-smi shows ~80% utilization.
• There is no speedup in training time compared to running on only CPU with tf.config.set_visible_devices([], "GPU").