Will Quant Finance End Up Like Data Science

A subscriber wanted to know if quant finance will end up like data science where undergrads will be hired as the new normal. The short answer is no but the answer is a bit complicated. Quantitative finance started out like data science as a trendy degree where every quant was a full stack quant. As the quantitative finance industry matured the job got split into quant (also known as quant researcher or model developer), quant dev (implementation), and trader (business user). Data science has had a similar full stack start but in the last few years has really started to segment into data engineering, data science, ML engineer, and dev ops. Even these new categories seem undecided on what they really mean. Quant finance is more mature as an industry that data science however BOTH still have a long ways to go.

Website:

Support:

Quant t-shirts, mugs, and hoodies:

Connect with me:

[winggambit]

Data science nowadays can mean anything between making pizza and proving deep statistical theorems

[ABSTRACTSHNITZEL]

Thanks for answering my question, Dimitri. This clears up a lot of the things I was wondering. It seems like a Master's degree isn't "the new bachelor's degree" but rather jobs have hijacked titles that used to have Master's degree requirements.

[theJasonLee]

This is sooo validating to hear!! So frustrating joining a team (or interview) and being asked ridiculous questions about overly complicated 'trendy' models or packages, and essentially being forced to ask why they don't do something much more straightforward, simple, and maintainable. Very hard to always be providing that feedback. Folks don't want to hear it!

[jasonavina]

Hey great video, I had to comment, because currently I'm a graduate student in a Masters of Data Science and Artificial Intelligence. So everything you're talking about is something I think about almost daily. The split you describe is even apparent in my university, where there is a program in the CS department(my program) and one in the Math Department called "Statistical Data Science". And my program really pushes the ML and programming aspect, but I'm finding that a lot of my classmates don't really understand how the models work on a mathematical level, and it makes me really skeptical. So my focus in my program has been to shore up my statistical skills as much as possible and fill up my courseload with as many Mathematical Statistics, model development, and higher level statistics classes as possible. It strikes me as really strange that a programmer would know how to program A.I. to solve a problem, yet can't do a basic linear or logistic regression and it makes me a bit uneasy. So my goal is to avoid that if possible. I am teaching myself finance by the way since my undergrad was not in the financial field, and am reading books like "Quantitative Financial Analystics" and "Option Volatility and Pricing"(with the workbook :) ).

[parhamhamouni]

Thank You! Finally, someone said it as it is. I am a data scientist and I hate the way it has evolved.

[vaibhavmalviya]

Another issue that can be seen LinkedIn job postings is that majority of Data Scienctist jobs are strictly just Data Engineering jobs, and very few actual model building jobs.

[nishu]

Separating curve fitters from actual data scientists is the industry equivalent of separating men from the boys. It’s kinda needed tbh. I’m a statistician turned data scientist and now looking to transition into the quant space. I feel your frustrations, Dimitri.

[royaltydeal]

If Data Science/ Data scientist was design for the purpose of making business environments more efficient using and interpreting data, How is it that they could choose to ignore Econometrics as a tool applied to economic and financial theory? Applied Econometrics to economic theory, financial theory and the business environment is essential in order to interpret the data in a business sense, otherwise your just operating like a chicken with it’s head cut off, doing a whole bunch of nothing. It seems to me that data science/scientists are more caught up with the trend rather than the scholarship of being someone who is train to solve real-world problems.

Thanks Dimitri for your input on this topic.

[nielubiegdyktospatrzyjakje]

That's actually good problem You've pointed out. I got hired in credit risk department in a large consulting firm. They're naming themselfs as the quants but the tools being used are precisely data science'ish. Mostly classification algos. Of course, they're not working on investment risk, it's just a banking, thus sophistication of tools being used is largely limited by regulation entities. But still. I'm undergrad with a DS course done, after that I've enrolled Financial Maths MSc, but offer was too good too wait for the end of the MSc. IMHO situation on the market is that it is better to hold on a decent job offer rather than pursuing masters, but it's a good idea to acktually pursue it finally and I'm sure about this.

[ruoxuanzhu]

Hi Dimitri, can you make a video about how the new Chatgpt4 can impact or facilitate different types of quant?

[allisterblue]

An issue when modeling phenomena which evolve through time is that you can't know for sure it won't blow up. My understanding is that, if you try to do forecasting, you either have a set of variables serving as a state which captures how your response evolves, or you have to rely on things like trend extrapolation, which feels like a gamble. The first option is ideal but rarely attainable, and for the second option, I'm not quite sure how you could avoid the pitfalls.

[pb]

Hi Dimitri
Have you too noticed that most who claim to work as data scientists lack the mathematical background to work on the same projects?
Are you saying that data science roles will sort of bottom out in 20 years and then pick up?
Given Quants have a more structured approach to problems and model building, do you believe it to be the reason why we still do not have an option pricing model based on machine learning?

[justinpardo-mwwy]

yeah good take i remember going to a hackethon meeting and one of the participates who identified who was a data scientist it was honestly a data analyst role mostly python some data pipelines and tableau.

[aanchitnayak]

I'm a consultant who did his master's in Operations Research. I also work with Corporate Finance as a domain.

I agree that the democratization of data science has led to a general degradation of discourse in the space. I can not believe I got hired simply because I could clearly explain a linear regression. The bar is low if you have been rigorous in your academic life. I have seen software devs who do data science not be able to explain what would be basic reasoning taught in an econometrics course, but will use the OpenAI APIs to sell their deliverables. It's sad.

[dopamine]

Hey do you have a video on job opportunities for people who just have or want a undergraduate degree?I understand quant jobs are out of reach (b.s economics minor in math, know python and have used it in internships)

[rachellee]

Hi Dimitri! Thank you for your video :) I recently got admission from University of Maryland with Quantitative Finance masters program. Do you think is it helpful to get a job as a quant in the United States? I’m an international student from Korea 😂

[aryamanmishra]

I am an undergrad who has taken courses like graduate string theory, quantum field theory, have experience with stochastic calculus and probability theory. I don't come from school per se but has a super famous theoretical physics department (Yang institute) and I worked with many of those Professors. How do I approach a company with a preparation but not much life maturity? I have lot of coding and some ML skills.

[jaykay]

The fact data scientists do not test hypotheses like econometricians do does not mean they do not have any specific questions they want to answer. Econometrics and statistics are like, I think or believe, or theory says A affects B, I want to test that hypothesis. Data science is more like, I don't have a theory, but I want to predict the future value of A...using several hundred or thousands of variables that I think affect A. Eventually, unimportant features would be dropped or given less weight. An econometrics model that can be tested does not necessarily mean it is a better predictor. And if you have hundreds or thousands of features with no formal theory, and prediction as the goal, maybe ML models are a better choice. What you have done in this video is trying to put econometrics/statistics models above data science models in quantitative finance. All your negative comments were on data science models, and all your positive comments were on econometrics/statistical models. These are two completely different models, doing different things. One should not try to compare them and choose one over the other. Econometric models want to estimate and test an effect given some predefined theory or hypothesis. Data science models, on the other hand, are about prediction when you have a lot of features with no formal predefined theory or hypothesis. If you are gonna criticize the data science field because of fake data scientists, why not also criticize the econometrics/statistics field because of fake econometricians and statisticians? Because there are fakes in any field, it does not matter whether data science or econometrics or statistics. I agree, though, data science has become easier, but so is econometrics and statistics. Defining a hypothesis is not rocket science...and anyone can come up with a testable hypothesis and use statmodels or STATA, SaaS or R, etc. One reason why data science has become popular (and not econometrics or statistics) is that you can find new insights from massive data with no predefined theory or hypothesis. But in your view, I guess that advantage is rather a flaw and not an advantage. Well, I would say do not compare oranges to apples. Besides, I also think data science has become popular because the resources to do data science are free...unlike econometrics/statistics in the past, where you needed paid software (SaaS, MATLAB, STATA, EVIEWS, SPSS, etc.). If these were free in the past, maybe, econometrics/statistics would have been like data science, where everyone can do it with almost no cost. Your trashing of the data science field here is largely unjustifiable because the same arguments also apply to the econometrics and statistics fields. In any case, testing a hypothesis is no rocket science...and it can even be easier than looking for meaningful insights in a huge data set. Just a thought. I am a fan of your channel. .... an economist trying to switch to quantitative finance.

[googlegoogle]

Honestly all of this is just playing with language. Of course real Data Science requires rigorous scientific thinking and solid software engineering. And it's the same for Quant Finance. Just ignore all the fake it till you make it people that think they can work in a technical role without understanding what they are doing. Imagine a mechanical engineer developing critical parts of a car without understanding the models...

[janulation]

You're renting about data scientists (part of that is true) but then misuse some terms your self. Like you put ML and AI together...ML is a subset of AI by definition.