Guest Post: How Mava Uses Langfuse to Improve their Usage of OpenAI's API
by Richard Draper (opens in a new tab), co-Founder of Mava (opens in a new tab)
About Mava
Mava (opens in a new tab) is an AI-first customer support platform built for community-first organisations. Hundreds of SaaS, Gaming, Web3 and open-source projects already use Mava to help support their communities.
Mava has an AI bot that can auto-respond to commonly asked questions in public community channels, such as Discord as well as a full omni-channel support dashboard for your whole team.
Why were we looking for a solution like Langfuse?
Like many other companies, we quickly recognized the limitations of utilizing off-the-shelf OpenAI APIs when deploying, operating, and trying to improve LLM outputs. It was hard to understand where improvements could be made, get a clear picture of usage and performance and then track this over time.
Why we use Langfuse at Mava
Langfuse has removed a huge amount of guesswork and opacity from the process of monitoring and improving LLMs and saved us a lot of time. Langfuse has been especially useful in three areas. 1) helping us improve LLM output, 2) debugging issues and errors and 3) providing more clarity on cost, usage and latency.
Improving LLM output
We needed a solution that allowed us to evaluate and fine-tune changes to our language models instantly. Langfuse scoring provided us with a robust and scalable approach to assess the quality of iterations. It enables us to immediately see and test changes, such as prompt adjustments. Traces are also useful to understand exactly how our bot reasons.
Debugging issues
Langfuse's Traces feature proved invaluable for us in swiftly identifying and resolving issues. With instant insights into the root causes of problems, we can streamline our debugging process efficiently - it has saved us a lot of time.
Analytics, latency, and costs
The analytics dashboard is fantastic and provides a much greater level of granularity compared to the likes of OpenAI. This helps the development team and other team members to easily keep track of metrics that matter to them. The pre-built dashboard is excellent and it’s easy to drill-down into the data in more detail.
What we love about Langfuse
Within a few days Langfuse became an indispensable tool, saving our development team a lot of time and enabling us to make significant improvements in our LLM output. The Langfuse team is also exceptionally responsive (opens in a new tab), open to feedback and ships at lightning speed. Importantly, they clearly take security and privacy seriously, with a robust infrastructure that includes features like data tenancy which was an important prerequisite for us at Mava before interacting with 3rd party vendors. Finally, it was super quick for us to integrate with Langfuse.