Lecture 14B: Modern Private ML - Differentially Private Stochastic Gradient Descent

Показать описание

Рекомендации по теме

Комментарии

Amazing! I have a simple question:
Why we can use composition lemma to handle the total privacy? I think each iteration is not independent.

ufsbdubf

Hi, thank you for publishing these lectures (non privately :) ), super helpful! I had a query regarding gradient perturbation techniques. While I see adding noise to the gradient at every iteration privatizes the model, why would I be required to do so unless I want to publish each of my gradients? Suppose I just want to publish my model, and say I'm using cross entropy loss with neural networks, so convexity doesn't hold and therefore output perturbation is non-trivial - still, can't I just add noise to the final (or final few) gradients in the training procedure and go onto publish the model? I'm guessing I'm overlooking something here, since intuitively my final iteration is likely only to be a function of a small fraction of the data, and DP is post processing invariant, but surely can't undo sensitive information captured apriori; but I can't quite get my head around formalising why this wouldn't be sufficient? Apologies in advance if this is really silly

dronakhurana

This is amazing. I have a simple question on the sensitivity of the DPSGD method.
Why is the L2 sensitivity for adding noise C. It should be sqrt(2C^2) for neighboring datasets. Why was 2 removed? This is with reference to the Deep Learning with Differential Privacy paper referenced in your CS860 course
Thanks

iyiolaolatunji

I was wondering if there were an underlying assumption for gradient perturbation, that having the gradient equals seeing the data. Yet I don't see anyone talk about this. But my guess is that if gradient cannot fully represent the data, there is already some inherent privacy, i.e. releasing it is not blatantly non-private.

junyizhu

Is it equivalent to adding N(0, sigma/L) to each individual of i in the lot?

GundamCipher

Lecture 14B: Modern Private ML - Differentially Private Stochastic Gradient Descent

Lecture 14B: Modern Private ML - Differentially Private Stochastic Gradient Descent

Lecture 14A: Modern Private ML - Some Background on Neural Networks

Lecture 14C: Modern Private ML - PATE

Lecture 14B: Explaining Decisions (PI Explanations, Sufficient & Complete Reasons)

Concentrated Differentially Private Gradient Descent with Adaptive per-Iteration Privacy Budget

Antti Honkela: Accurate privacy accounting for differentially private machine learning

430 - Adaptive Privacy Preserving Deep Learning Algorithms for Medical Data

Lecture 13C: Differentially Private Machine Learning - Gradient Perturbation

Fast and Memory Efficient Differentially Private-SGD via JL Projections

Privacy, Stability, and Online Learning

Interpretation of the ε and δ s of Differential Privacy | Lê Nguyên Hoang

Differential privacy dynamics of noisy gradient descent

Research talk: Differentially private fine-tuning of large language models

DP-SGD Privacy Analysis is Tight!

The Mathematics Behind Differential Privacy

Differential Privacy from a Statistical Perspective–Obtaining Valid Inferences...

Session 3C - Private Stochastic Convex Optimization: Optimal Rates in Linear Time

Private-kNN: Practical Differential Privacy for Computer Vision

Differentially Private Model Publishing For Deep Learning

KDD 2020: Lecture Style Tutorials: Edge AI Systems Design and ML for IoT Data Analytics

The Formal Definition of Differential Privacy | Lê Nguyên Hoang

Mathematics for ML: Lecture 4

OSDI '20 - Fast RDMA-based Ordered Key-Value Store using Remote Learned Cache

Genius! Can you answer this?🤔 | Maths Challenge | CBSE Class 7 | Anushya Mam #shorts #ytshorts