Integrating Fluent Bit Into a PaaS - Bojan Djurkovic & Dhruv Kela, DigitalOcean

Integrating Fluent Bit Into a PaaS- Bojan Djurkovic & Dhruv Kela, DigitalOcean

At DigitalOcean App Platform we use Fluent Bit for log processing of internal components and for customer-facing functionality. This session will review how we collect internal logs from dozens of Kubernetes clusters into a centralized logging system utilizing a custom plugin. We will also describe how we dynamically configure Fluent Bit for customer applications' log processing. This presented several operational, scalability, and security considerations. Orchestrating Fluent Bit in a security-restricted environment proved challenging. We also need detailed observability into applications' logging pipeline health and metrics. Different implementation approaches and potential future improvements will be discussed.