InvalidArgumentError Unsupported 'dtype' Value bfloat16
Quick answer
This error means the specific TensorFlow op you called does not accept bfloat16, even though TensorFlow supports the type generally — kernel support is registered per op, and many ops only allow uint8, int32, int64, half, float, or double. Cast the tensor to tf.float32 before the op with tf.cast(x, tf.float32), then cast back afterwards if you need bfloat16 downstream.
InvalidArgumentError: Value for attr 'dtype' of bfloat16 is not in the list of allowed values: uint8, int32, int64, half, float, double
In TensorFlow, when creating tensors or specifying the data type for operations, you need to choose from a list of supported data types. The 'dtype' attribute specifies the data type of a tensor or operation result.
The error message "InvalidArgumentError: Value for attr 'dtype' of bfloat16 is not in the list of allowed values: uint8, int32, int64, half, float, double" indicates that 'bfloat16' is not a valid data type in TensorFlow. It is not included in the list of allowed values.
To resolve the issue, you should select one of the supported data types and use it instead. Let's explore each of these data types and provide examples:
-
uint8: Unsigned 8-bit integer. Example usage: Image data with pixel values ranging from 0 to 255.
-
int32: 32-bit integer. Example usage: Labels or indices in a classification task.
-
int64: 64-bit integer. Example usage: Large integer values or indices.
-
half: 16-bit floating-point. Example usage: Memory-constrained environments that benefit from reduced precision.
-
float: 32-bit floating-point. Example usage: General-purpose floating-point operations.
-
double: 64-bit floating-point. Example usage: High-precision calculations requiring increased accuracy.
When creating a tensor or specifying the 'dtype' attribute in TensorFlow operations, ensure that you select one of the valid data types mentioned above. Here's an example that illustrates the correct usage:
import tensorflow as tf
# Creating a tensor with dtype=float32
tensor = tf.constant([1.0, 2.0, 3.0], dtype=tf.float32)
# Performing a TensorFlow operation with the correct dtype
result = tf.math.sqrt(tensor)
# Printing the result
print(result)In this example, we create a tensor with a valid 'dtype' of 'float32' and then perform the square root operation. This code will execute without any errors.
However, if you were to replace tf.float32 with tf.bfloat16 in the above code snippet, you would encounter the "InvalidArgumentError" due to the unsupported 'dtype' value.
Remember to choose an appropriate data type from the list of supported types when working with TensorFlow operations to avoid encountering this error.
Key takeaways
- •bfloat16 is a real TensorFlow dtype, but kernel support is registered per operation — the error is about one op, not the type as a whole.
- •The allowed list in the message (uint8, int32, int64, half, float, double) is exactly what that op's kernel accepts.
- •The standard fix is tf.cast(tensor, tf.float32) before the failing op, casting back afterwards only if needed.
- •bfloat16 has the same exponent range as float32 but fewer mantissa bits, so casting between the two never overflows — only precision is lost.
- •bfloat16 is designed for TPUs and recent NVIDIA GPUs; on unsupported hardware it brings no speed benefit and more compatibility problems.
- •Prefer mixed_precision policies over manual bfloat16 casting — Keras then handles dtype placement per layer for you.
Frequently asked questions
Why is bfloat16 'not in the list of allowed values' if TensorFlow supports it?
TensorFlow registers kernels per operation and per dtype. An op only accepts a dtype if someone implemented and registered a kernel for that combination. Many ops, especially older or less common ones, have no bfloat16 kernel, so passing one raises InvalidArgumentError even though the dtype itself is valid.
How do I fix the error?
Cast to a supported dtype before the failing operation, normally float32: `x = tf.cast(x, tf.float32)`. Run the op, then cast the result back to bfloat16 if the rest of your pipeline needs it. Casting from bfloat16 to float32 is lossless.
Will casting to float32 hurt my model's accuracy?
No. bfloat16 has the same 8-bit exponent as float32 but only 7 mantissa bits, so widening to float32 loses nothing — it only adds precision. The cost is higher memory use and possibly slower execution for that operation.
When should I actually use bfloat16?
On TPUs and recent NVIDIA GPUs (Ampere and later), where hardware support makes it meaningfully faster and halves memory use with almost no accuracy loss for training. On older hardware there is no speed benefit, so it adds compatibility risk for nothing.
Is there a better approach than casting manually everywhere?
Yes — use Keras mixed precision: `tf.keras.mixed_precision.set_global_policy('mixed_bfloat16')`. Keras then chooses dtypes per layer, keeping numerically sensitive operations such as softmax and loss computation in float32 automatically.