In today's data-driven world, having access to real-time data is crucial and gaining a value from the data is more cumbersome. Another challenge developers face when working with real-time data is efficiently and quickly fetching, processing, and displaying the insights from the data.
Enter Dozer, it simplifies the process of creating low-latency data APIs (gRPC and REST) from any data source easily with OpenAPI specification. Dozer is a real-time data engineering tool built with Rust, which enables seamless integration of data streams(sources), efficient data processing, and lightning-fast response times. This allows developers to focus on building applications without worrying about the complexity of managing real-time data.
To know more about it's capabilities and features you can checkout Dozer repository.
In this blog, we'll walk you through the process of creating a real-time flight data application using Dozer, React.
For this particular blog, we are using PostgreSQL as our data source, but it's important to note that there can be multiple data sources that can be connected to Dozer and subsequently used by your React application. By using Dozer, you can quickly create efficient data APIs that enable seamless integration with your frontend applications.
PostgreSQL is connected to Dozer via PostgreSQL connector and fetches data from multiple Postgres tables that has a flight data and combines them in real time based on the queries and produces fast READ APIs.
On the client side, React app is being used to display the flight data on map coming from Dozer. The dozerjs
and dozer-react
libraries are used to make it easy to interact with Dozer and access real-time data without having to hardcode API endpoints in the code.
The dozerjs
library manages gRPC requests between the frontend and the Dozer app, while the dozer-react
library provides React hooks for easy integration with React components.
So let's dive in and learn how to harness the power of Dozer to create amazing real-time data-driven applications! πͺβ¨
π Setting Up PostgreSQL:
- Clone the sample repository and navigate to the appropriate directory:
git clone https://github.com/getdozer/dozer-samples.git
cd dozer-samples/usecases/pg-flights
- Checkout the
react
branch:
git checkout react
- In the
usecases/pg-flights/
directory, run the following command to start PostgreSQL using Docker Compose:
docker-compose up
This command will create a PostgreSQL container with the necessary configuration and sample flight data as shown below:
To test the connection to the PostgreSQL container, you can use psql
or any PostgreSQL client. Here's an example using psql
:
psql -h localhost -p 5437 -U postgres -d flights
This command connects to the flights database using the postgres user on the default port 5437
. Once connected, you can run SQL queries to explore the sample flight data.
With PostgreSQL set up and running, we can now proceed to build the React application and configure Dozer.
βοΈ Building & Running the React App π:
- Open a new termincal and navigate to the
/usecases/react
directory in the cloned dozer-samples repository:
cd dozer-samples/usecases/react
This directory contains a sample React app that displays flight data on a map component.
- Install the required packages for the app using yarn install:
yarn install
- In the project directory, run the following command to start the development server:
yarn start
This command runs the app in development mode.
Open http://localhost:3000 to view it in your browser. The page will automatically reload when you make changes, and you may also see any lint errors in the console. On this page, you will the Dozer logo, now navigate to http://localhost:3000/airports to see the map UI for respective ariport and it's flight booking data as follow:
This React app uses the @dozerjs/dozer and @dozerjs/dozer-react libraries to interact with Dozer, enabling seamless integration and real-time data updates.
Since, we have not configured the Dozer with PostgreSQL yet, there is not data on the map.
Let' quickly setup the Dozer and see the magic happening!
π Configuring Dozer and PostgreSQL Connector:
- Start by installing Dozer. In for this blog, we'll use Homebrew for the installation. To learn about other installation methods, visit the official Dozer installation documentation.
open a new terminal, ensuring you're in the same directory as your React project, and run:
brew tap getdozer/dozer
brew install dozer
- Once the installation is complete, create a
dozer-config.yaml
file in your project directory and paste the following configuration into it:
# dozer-config.yaml content
app_name: flight-microservices
connections:
- name: flights_conn
config: !Postgres
user: postgres
password: postgres
host: 0.0.0.0
port: 5437
database: flights
sql: |
select f.arrival_airport as airport, a.coordinates as coordinates, COUNT(t.ticket_no) as tickets
INTO airports_count
from tickets t
join ticket_flights tf on t.ticket_no = tf.ticket_no
join flights f on tf.flight_id = f.flight_id
join airports a on f.arrival_airport = a.airport_code
group by f.arrival_airport, a.coordinates;
select extract(HOUR FROM f.window_start) as start, count(f.window_start) as dep_count
INTO departures_count
from TUMBLE(flights, scheduled_departure, '4 HOURS') f
group by extract(HOUR FROM f.window_start)
sources:
- name: tickets
table_name: tickets
columns:
connection: !Ref flights_conn
- name: flights
table_name: flights
columns:
connection: !Ref flights_conn
- name: ticket_flights
table_name: ticket_flights
columns:
connection: !Ref flights_conn
- name: airports
table_name: airports
columns:
connection: !Ref flights_conn
- name: airports_flights_schema
table_name: airports
columns:
schema: flights_schema
connection: !Ref flights_conn
endpoints:
- name: tickets
path: /bookings/tickets
table_name: tickets
index:
primary_key:
- ticket_no
- name: flights
path: /bookings/flights
table_name: flights
index:
primary_key:
- flight_id
- name: airports
path: /bookings/airports
table_name: airports
index:
primary_key:
- airport_code
- name: ticket_flights
path: /bookings/ticket_flights
table_name: ticket_flights
index:
primary_key:
- ticket_no
- flight_id
- name: airports_count
path: /airports_count
table_name: airports_count
index:
primary_key:
- airport
- coordinates
- name: departures_count
path: /departures_count
table_name: departures_count
index:
primary_key:
- start
This configuration defines the PostgreSQL connector and sets up the API endpoints that our React application will use to fetch data.
For more information about the PostgreSQL connector, refer to documentation.
Dozer supports multiple connectors, allowing you to integrate various data sources. Check out the list of available connectors for more options.
- With the configuration in place, run Dozer in action by executing the following command:
dozer -c dozer-config.yaml
Now, you should see Dozer logs indicating that the endpoints is ready to be queried by the React app.
You can now access your application at http://localhost:3000/airports to see the real-time flight data populating on the map component as shown below:
Additionally, You can also query the API using gRPC and REST endpoints to test the data. In this case, you can use the REST endpoint using curl or Postman http://localhost:8080/airports/
Additionally, you can check the OpenAPI specification documentation that Dozer generates for endpoints by requesting http://localhost:<port>/<endpoint_path>/oapi
as shown below.
π Conclusion:
Congratulations! You've successfully built a real-time flight data app using Dozer, React, and PostgreSQL. This powerful combination enables you to create interactive and data-driven applications with ease.
Feel free to expand on this example and explore other data sources and connectors to enhance your app's capabilities. Enjoy exploring the application and customizing it further to suit your needs!
Happy coding, Happy Data APIng! ππ©βπ»π¨βπ»
Top comments (0)