How Flibco leverages the Modern Data Stack

markus-krisetya-Ifs-BFxjdJ8-unsplash

Being a leading player in the niche of airport transportation with ambitions to expand more internationally, requires Flibco to anticipate new changes: new technologies and all social aspects. This means steering that ambition towards a single goal, which is
to continually improve people’s daily lives.

To succeed and provide a way for their customers to move around independently, Flibco is continuously striving to develop sustainable solutions and innovative products and services that respect the environment. Creativity and discovery, guided by a sense of responsibility, drive them forward. Forever pushing ahead.

Flibco connects the dots from your city to the airport. Check out our various
departure destinations. Using real time flight times, we make sure to pick you up exactly when needed.

Flibco travels green. Instead of using your own car, you can help in reducing the amount of vehicles on the road. Travelling together will help to minimise the impact on the environment. Fair-priced, Flibco.com gives you time to yourself to do exactly what you wish, listen to music, sleep, or work. Enjoy a comfortable journey with plentiful legroom and space for any type of luggage.

The Goals

Flibco is a Transportation Management Software, custom developed, which runs on a semi-structured (NoSQL JSON) database.

With flibco being operational for certain time, they were looking to extract maximum value and insights from the underlying datasets the platform generates. These data-driven insights would enable Flibco to make impactful business decisions with fact-based answers from their data such as:

Productivity

How much time do drivers spend actually driving around passengers?

Compliance

Does the scheduled vehicle of any given trip comply with specific passenger needs? (e.g. disabilities, child seats, etc.)

Human Resources

How much overtime did a driver perform in any given month? What does that mean in terms of salary calculation?

Sales

How is Flibco performing against their KPI’s for bookings, tickets, trips, etc.?

The Challenge

Removing data silos to increase data clarity
When making impactful data-driven decisions, you first need to have the full picture in front of you. Bringing data together from various data sources, allowed Flibco.com to better connect the dots between different data sets and build an holistic view of the software.

Data transformation & modelling to increase data quality
Getting easier, faster answers from data, required Flibco.com to significantly improve the quality of their data first. Flattening the JSON datasets, handling different UTC timezones, and achieving some level of modularity in the data were among the bigger challenges of Flibco’s data journey.

Driving a company-wide understanding of data flows
When people work with data, they need to trust the data they’re seeing. And that requires people to understand what data they’re seeing, and where it's coming from. Troubleshooting at Flibco.com was difficult due to lack of both proper documentation and a company-wide understanding of data flows.

Accelerating the Flibco Data Journey with
the Modern Data Stack

To accelerate Flibco's data journey & increase the level of data maturity, a modern data stack infrastructure was implemented to allow for better, faster & more effective data-driven decision-making at scale.

Extract & loading data in under 5 minutes with Fivetran

Fivetran manages data delivery from source to destination with fully managed connectors, so Flibco didn’t have to worry about engineering.

Fivetran offers 150+ zero-configuration connectors that launch in minutes, allowing Flibco to load data into their data warehouse without hassle. Centralising all their data and empowering their team with faster insights.

Removing Data Silos with Snowflake

Snowflake’s support for data warehousing and analytics provides a low-maintenance, cost-effective way for Flibco to consolidate all their data silos into a single source of truth they can query to get results fast.

This streamlines data engineering, delivering performance and simplicity so Flibco and their data teams can now focus on getting value from their data instead of managing the pipelines and infrastructure. Win-win.

Data transformation within the warehouse with dbt

Dbt is a SQL execution tool that helps Flibco.com apply DevOps best practices to data transformation workflows.

It does not extract, hold or write any data; it does not even provide the CPU for the transformations. Now, it simply holds Flibco’s data logic and transformations (written as SQL scripts) and sends them to the (usually Cloud-) database for execution.

Intuitive analytics & data exploration with Tableau

Tableau offers robust data connectivity options, allowing users to connect to various data sources, from databases to cloud platforms, ensuring flexibility in data integration.

So, once placed on top of the Snowflake data warehouse, Tableau helped Flibco’s data users visualise and explore all of their data without having to write code. They can now dive deeper, pivot analysis, and change chart types without starting over or breaking their flow.

"Biztory's support in implementing our Modern Data Stack was crucial. They helped us centralize our data in Snowflake, streamline our data transformation processes with dbt, and make data more accessible through Tableau. Their proactive approach and technical expertise have allowed us to maximize the value of our data, driving both operational efficiency and innovation across Flibco.com."

Jan Weber, IT Project Manager at Flibco

Success factors...

Saving big costs with Fivetran

Having a standard connector available out-of-the-box for both the semi-structured source and the Snowflake destination, Fivetran was set-up in a matter of hours.

Almost no expertise was required from Biztory’s side. Fivetran’s tool was simple enough for Flibco’s internal colleagues to handle without too much trouble, which means the operating costs are based almost entirely on the tool’s usage, rather than Data Engineer FTEs.

Big cost saver.

Snowflake eats JSON for breakfast

With Snowflake’s all-powerful lateral flatten() function we had the JSON arrays and objects normalised in no time.

Being the CPU powerhouse that it is, Snowflake was also able to reduce runtimes of flows that used to take over an hour, to under 5 minutes. This without even requiring the Medium or Large “warehouses” (=CPU pools), so essentially in first gear.

Under the hood, snowflake stores all timestamps as an amount of milliseconds since 01-01-1970 in UTC time (as do many databases). By setting sessions or account timezones, Snowflake is then able to both present the actual time in the specified timezone (by adding or subtracting the required amount of hours from the UTC value), as well as
perform datediff() calculations on the actual UTC values.

Eg Snowflake knows the difference in time between 01:00 AM on October 29th 2023 (=UTC + 02:00) and 01:00 AM on October 30th 2023 (=UTC + 01:00) is 25 hours and not 24!

Fast and rigorous progress enabled by dbt

As mentioned above, dbt does not extract, hold or write any data; it does not even provide the CPU for the transformations. It simply holds Flibco’s data logic and transformations and sends them to the Cloud-database for execution.

In this pretty straightforward capacity, it is however capable of transforming Flibco’s Data Modeling flows to true DevOps-proof Data Engineering!

Some of the areas where dbt delivers value are:

DML/DDL & Materializations
Data Lineage & Documentation
Troubleshooting & Tracking Upstream Data
Scheduling
Advanced Coding
Automated Testing & CI/CD
Version control & DEV/PRD Environments

Conclusion

In summary, Flibco's adoption of a Modern Data Stack infrastructure, featuring tools like Fivetran, Snowflake, dbt, and Tableau, has significantly enhanced their data handling capabilities. This transformation allows them to make informed, impactful business decisions and continues to propel them as a driving force in the mobility sector of tomorrow.