Scikit Learn Machine Learning Tutorial for investing with Python p. 6

In this machine learning with Python and scikit-learn tutorial video, we cover how to use the Pandas module to help structure and modify our data.

No Pandas? No problem:
or

Bitcoin donations: 1GV7srgR4NJx4vrk7avCmmVQQrqmv87ty6

[JamesKao]

Some of source files are have a newline between the closing '</td>' and '<td class="yfnc_tabledata1">'.

Other files are missing 'Total Debt/Equity (mrq)' altogether.

Here's how I resolved it:
```
if gather + ':</td>' in source:
value = source.split(gather + ':</td>')[1].split('<td
else:
value = 'N/A'
print ticker + ':' + value
```

Awesome tutorial thus far Harrison!

[leocelente]

WOW! 40 seconds! That took me 5 minutes in 30% of my FX-6100, I bet you're using an i7, those things are productivity MONSTERS!

And by the way thanks for the tutorials, great content and simplicity. YouTube needs more people like you.

[kharicousins]

I've been looking for machine learning tutorials for Python lately, thanks =D

[cyl]

if anyone encounters any problems with 'decode', please fix this line:
source = open(full_file_path, 'r', encoding='utf-8').read()

I learned it the hard way. :)

[dushyantshukla]

What's the need of Unix time in our script??

[gauharaya]

if you are struggling to see your csv populated it is for the reasons described by @Saurabh below. Initialise your empty array like so -> data = []. Then do -> data.append({'Date': datetimeStamp, 'Unix': unixDTStamp, 'Ticket': ticker, 'DE Ratio': value}).

Outside the for loop convert your array of hashes into dataframe like so -> df = pd.DataFrame.from_dict(data). The rest is as @sentdex describes.

[blissfulgarima]

hey Sentdex, I just tried to do all the programming but ticker is not working properly it keeps showing me 'Users'instead of file names :-( and when i saved my csv file at the end it got saved but after a while i checked i did lil mistake inbetween so columns in csv file are not coming properly now i hv corrected my code and trying to save in same CSV it is showing me permission denied :-( . Is there any way out?

[张翼-fu]

I had no Programming language experience and start watching ur video after taking python class in codecademy. These are really great videos and I learned a lot from here, thx very much.

May I have a basic question: When I forget to type "df = " in code:

"df = df.append({'Date':date_stamp, 'Unix':unix_time, 'Ticker':ticker, 'DE Ratio':value, }, ignore_index = True)"

nothing is in .csv file produced, why?

[brosales]

Oh man, you make it looks easy, great Job.

[ronniec]

Just a thought but maybe try SortedContainers for your data append. It'll be faster than Pandas as it wont do the dataframe copy (which it does on an append), and then finally convert to a dataframe at the end; which is a copy(sadly)

[dorsolomon]

What do you mean be labeling the data?

[TheTaquitoProject]

how to scrape old data from yahoo finance?
what url can I find said data?

[Kattarhume]

I am getting an empty dataframe by doing so, is there any other way to add a row to dataframe other than append or using dictionary as shown in video.

[nehmajmudar]

I am getting an empty dataframe (.csv) file, except the column names. Please help.

[MrAsardi]

thanks for the video. one question, at the end I run the file, now where exactly is my saved csv file?
I cannot locate it for the love of thanks.

[rafalpilat]

If i look in the CSV file, in my file, the ticker starts at rok and other companies like appl arent even there. Any ideas why this might be the case? (Im on Ubuntu so that might be the problem)

[darronpressley]

Hi Sentdex

Is your code for this available to download, I have typed it in and get 

    value = source.split(gather + ':</td><td
IndexError: list index out of range

I know its probably a typo but I am going cross eyed trying to find  it.

[izaccy]

using Mac, Anaconda gets this error.

ticker = each_dir.split("\\")[1]
IndexError: list index out of range