Learning the tricks of Grafana Loki for distributed logging at scale in a Kubernetes environment

Logging can provide immense detail when used well, or it can become a firehose and take hours to trawl through. The team supporting the Kubernetes platform at Civo needed a solution that was simple and performant and could be queried in ways to help and not hinder them

In this talk, Civo SRE Anaïs Urlichs and Principal Engineer Alex Jones will illustrate how Loki was chosen and brought into the organization to empower engineers. Integrating with Prometheus and Grafana dashboards, Loki has allowed engineers to filter for precise information that helps them debug quicker.

The distributed nature of the tooling enabled the team to scale components to meet demand and configure it for Civo’s specific cluster topology.As part of this talk, Urlichs and Jones will demonstrate how you can integrate Loki into your Kubernetes ecosystem and quickly start pulling relevant information. They will also highlight the easy-to-use Explore functionality within Grafana and the advanced LogQL language to help find the right logs.

[ovidisai]

Hi, great vid, do you have also an example GIT repo with the example config files?

[musaermeydan]

First of all, thank you for such great work. However, some texts, especially the commands entered in the Grafana Dashboard, cannot be read clearly.

Also, it would be even more efficient if you shared a git repository link of the codes you used.

[rogercorpin]

Great video. Btw, do you have the source code of this tutorial? Thanks a ton!

[ranajitjana]

I did not understand the Graphics question. Is the name which was told was Excalator or Kudos. Could not catch it.

[sfincione]

It's almost impossible to see the text when you do the code demo part... it's way too small.

[RebeliousSapien]

i still don't get what the ingester is doing with the arrow pointing to the querier.
the querier is reading from storage, i get that
but what is the ingester doing connected to the querier? .. what request is making that happen and what is it for?

[de-is]

I hate when so called "tech gurus", "tech presenters", can't figure out their mic setup, so it sounds clear and pleasant!

[ervikasrastogi]

Hi....I have loki promtail and grafana setup and want to see grafana usage per user with dashboard from grafana.log..I have referer label in logs which is having dashboard name along with https url but i only want to show dashboards accessed userwise..can you please help me for same?