InterviewsVector

How to Change default types globally in tensorflow?

TensorFlow & GPUs

Quick answer

Use tf.keras.backend.set_floatx('float64') to change the default float type Keras uses for layer weights and computations. For mixed-precision training, use tf.keras.mixed_precision.set_global_policy('mixed_float16') instead. ConfigProto does not control data types — it configures session and device options only, so setting a floatx attribute on it has no effect.

TensorFlow has no single global "default dtype" switch. What you set depends on whether you want higher precision or faster training.

Set the Keras default float type

import tensorflow as tf
 
tf.keras.backend.set_floatx('float64')
 
layer = tf.keras.layers.Dense(10)
print(layer.dtype)      # float64

Check or reset it:

tf.keras.backend.floatx()          # 'float64'
tf.keras.backend.set_floatx('float32')   # back to the default

Call this before building layers — existing layers keep whatever dtype they were created with.

ConfigProto does not control dtypes

A common misconception. tf.compat.v1.ConfigProto configures the session: device placement, threading, and GPU memory. It has no dtype field:

config = tf.compat.v1.ConfigProto()
config.gpu_options.allow_growth = True     # valid — memory option
config.floatx = tf.float16                 # not a real field; does nothing useful

For speed, use mixed precision instead

Setting float16 globally is the wrong tool — softmax and loss computations overflow at that width. The mixed precision API handles the split for you:

tf.keras.mixed_precision.set_global_policy('mixed_float16')
 
print(tf.keras.mixed_precision.global_policy())
# <Policy "mixed_float16">

That runs most layers in float16 while keeping the variables and numerically sensitive operations in float32. On TPUs and recent NVIDIA GPUs, use 'mixed_bfloat16'.

With mixed precision, make the final layer's output float32 explicitly:

outputs = tf.keras.layers.Dense(10)(x)
outputs = tf.keras.layers.Activation('softmax', dtype='float32')(outputs)

Tensor creation ignores set_floatx

tf.keras.backend.set_floatx('float64')
 
tf.constant(1.0).dtype                    # float32 — inferred, not floatx
tf.constant(1.0, dtype=tf.float64).dtype  # float64 — explicit

Pass dtype= when the width matters.

Which to use

GoalAPI
Higher precision (scientific work)set_floatx('float64')
Faster training on modern GPUsmixed_precision.set_global_policy('mixed_float16')
Faster training on TPUmixed_precision.set_global_policy('mixed_bfloat16')
GPU memory behaviourtf.config.experimental.set_memory_growth

Key takeaways

  • •tf.keras.backend.set_floatx('float64') sets the default float dtype for Keras layers and weights.
  • •ConfigProto configures devices, threading, and GPU memory — it has no dtype setting at all.
  • •Call set_floatx before building any layers; existing layers keep the dtype they were created with.
  • •For performance, prefer mixed precision (mixed_float16 or mixed_bfloat16) over globally setting float16.
  • •Never set float16 globally via set_floatx — softmax and loss computations overflow; mixed precision keeps those in float32 for you.
  • •tf.constant does not follow set_floatx: literal Python floats still infer float32 unless you pass dtype explicitly.

Frequently asked questions

How do I change the default float type in TensorFlow?

Call tf.keras.backend.set_floatx('float64') before creating any layers. Keras then uses that dtype for layer weights and computations. Note this affects Keras, not tensor literals — tf.constant(1.0) still produces float32 unless you pass dtype explicitly.

Can I set default dtypes with ConfigProto?

No. tf.compat.v1.ConfigProto configures session behaviour — device placement, threading, and GPU memory options such as allow_growth. It has no field for data types, and assigning an arbitrary attribute to it does nothing.

What is the right way to use float16 for speed?

Use the mixed precision API: tf.keras.mixed_precision.set_global_policy('mixed_float16'). It runs most operations in float16 while keeping numerically sensitive parts, such as softmax and the loss, in float32, and keeps a float32 master copy of the weights.

Why shouldn't I just call set_floatx('float16')?

Because it forces everything to float16, including operations that overflow or lose too much precision at that width — notably softmax, loss computation, and weight updates. Mixed precision exists precisely to avoid that failure mode.

Does set_floatx affect tf.constant and tf.Variable?

Not directly. set_floatx governs the default Keras layer dtype. Tensor creation from Python floats still infers float32, so pass dtype explicitly — tf.constant(1.0, dtype=tf.float64) — when you need a specific width.

By Mohammad Wasi

Software Engineering Leader & Technical Author · Updated July 21, 2026

See all posts

Related Posts