Definition of a 'bit', in information theory

A link to the full video is at the bottom of the screen.

That video describes using information theory to write a bot that plays Wordle

Editing from long-form to short by Dawid Kołodziej

[diamantnt]

Can we just realize how big the square is with 1 terabyte

[johnchessant]

The full video is great. But, I think there's probably a better minute from it to use as a short than deriving the formula i = -log_2(p), e.g. clarifying what is meant by "observation" and "cutting the possibility space", or motivating why we define "information" logarithmically like this (because this way, successive observations just mean adding the numbers of bits)

[Aryeh-o]

The log terms are negative to cancel out the fact log2 of n < 2 also is negative .

if you cut options by 128 you use 7 bits of information, even if some of the space maps to the same value:

like when x=a with p=1/128 and x= b otherwise.

this is later used in the random forest algorithm which is fundamental to datascience.

[arf]

Good 'ol Shannon would be proud of this video! Now please do the whole theory. ✨🙏🏽

[paicemaster]

Do you have any other videos elaborating on information theory? The other stuff on youtube is just really hard to follow for me and i want to learn more.

[Hauketal]

And then there are derived units like Mb/s (Megabits per second) for data transfer or mb (millibit) for some peoples brain size. 😅

[shamikpatro]

Someone’s been reading Shannon & Tukey’s work 👀

[sabinrawr]

I would love to know why it would be negative. How to you have negative information?

[prajwalpai]

Can't you use the fact that there is no point that exists beside the pivot that does not come in contact with the line because the line covera the entire area

[michaellin]

And if you use the natural log instead, it's called a nat, not a bit

[wyboo]

does this imply that probability is not unitless, but has units 2^bits?

[mdandry]

How to destroy what reg people THOUGHT they knew about bits.

[gibcof]

I cant be the only one that has no idea what hes saying. Love his channel though

[marcomabr]

How many bits thos it fit in a micron and homay you need to fill 100 pounds

[Jimmy_Neutron]

I love how you discuss, but I can't understand any

[tomasseeber]

I believe sometimes you overcomplicate things.

- This comment is intended to give you an external point of view so you can work on it, which I'm sure you would be happy to, if you find it true, since it is evident that you have put a lot of work in your psyche, which I admire.

I = -Log2(p) was the objective, so the explanation and the steps are correct, necessary and clear, but, the objective was wrong.

If instead of probability we speak in terms of possibilities, we get rid of the fraction, and the need of a s negative logarithm.

P = 2^I
Then
I = Log2(P)

[event]

Let define "information" - of course information theory Has not any definition of information

[cbebutuoy]

awfully cavalier with your assumption about what “we“ call any of this stuff. I think you may be a bit off.