In an effort to enhance urban pet care, I explored the Open Data API provided by the City of Madrid to identify water points available across the city for pets, especially for dogs and cats. By integrating this data, I created an interactive HTML map showcasing the exact locations of these water stations, helping pet owners easily locate the nearest spots where their pets can drink water.
In the era of artificial intelligence, I invested in an in-depth examination of NVIDIA’s stock trends up until June 2024. By leveraging the Gemini API and advanced AI techniques, it was possible to observe and interpret stock trend charts over time with enhanced precision and insight.
In this research, I aimed at a streamlined architecture that can support a huge amount of data using NoSQL and Kafka inside a strange world which I don’t have much experience in the real world. The streaming pipeline is a little bit different from batch, however, at the same time is almost the same idea (at least for me).
I always heard about web scraping but I have no idea how or what will be the purpose of using it. That was really strange to me because in my projects I always used the corporate system as a source and didn’t use it in any project that I had worked on.
This pipeline provides a comprehensive demonstration of the end-to-end process, excluding the initial data source integration. It encompasses best practices in data modeling (star schema), leverages the power of Spark for distributed computation, and ultimately delivers a compelling outcome in the form of a detailed Power BI report.