Inside TensorFlow: Quantization aware training

In this episode of Inside TensorFlow, Software Engineer Pulkit Bhuwalka presents quantization aware training. Pulkit will take us through the fundamentals of quantization aware training, TensorFlow/Keras API used to achieve this, and how it is implemented during this tutorial.

[foolmarks]

GitHub link doesn't work. Audio is terrible.

[autripat]

Hey all, at 14:26, are we missing the quantize_annotate_layer wrapper over the Conv2d layer (inside Sequential), like this:
quantize_annotate_layer(tf.keras.layers.Conv2d(32, 5, input_shape=(28, 28, 1))

[athreyamurali]

Hey can you re-upload with better audio, please?

[shubhammane]

I tried QAT, as result I got .h5 model with quantize wrapper layers, I want to remove it and get back my original model with modified weights, How can I dot that?

[Havc]

Hey Pulkit,
Say I wanted to constrain quantization parameters to power of 2 values. Would that be supported?

[Lisa-hbjs]

I got this error whatever I do (the same if the network only contains Dense layers...) : ValueError: Unable to clone model. This generally happens if you used custom Keras layers or objects in your model. Please specify them via `quantize_scope` for your calls to `quantize_model` and `quantize_apply`. [Layer object at 0x7bedca8083a0> supplied to wrapper is not a supported layer type. Please ensure wrapped layer is a valid Keras layer.].

[PremKumar-qicd]

When I try to post-quantize(int8) the SimpleRNN model for a time series data, it is throwing an error saying only single graph is supported. So Does the RNN, LSTMs support for quantization and conversion to tflite models? And If yes, how can I address the error? Thanks in advance.:)

[sanjoetv]

i am having a problem when i convert my .h5 to tflite, , when i test the tflite on my mobile app the accuracy is so much lower than when i try to run the .h5 on jupyter.... my question is does quantization aware training can help me to lower the accuracy loss when converted it to tflite after the quantization aware training?

please someone help!!!

[lisali]

Thanks for sharing! Does it support mixed precision?

[anishdeepak]

I have trained ssd_mobilenet_v2 model using object detection api and saved the model as .pb file. How to apply the quantization to a my model. I dont have .h5 file.

[sreeragm]

Is there any scenarios in which quantisation shouldn't be done? Like, Incase I want to convert it to other formats supporting optimization, such as TensorRT.

[raisaalphonse]

I'm using QAT for a functional model only, but I'm getting a value error saying,

quantize_model
'`to_quantize` can only either be a tf.keras Sequential or '
ValueError: `to_quantize` can only either be a tf.keras Sequential or Functional model.



I'm not really sure why I'm getting this error. Could anyone please help me out in this?

[yoloswaggins]

Can this be used for tensor cores on Nvidia GPUs or is it only for embedded devices?

[rupeshmohanasundaram]

for object detection, QAT Works, if so how?

[sairamvarma]

The Github link in the description doesn't work

[travelsome]

Waiting for a video for sequential modelling

[bryanlozano]

Bruh, is someone weed-whacking outside?