30 min listen
"Saga of a Gnarly Report" with Owen and Dan
FromElixir Wizards
ratings:
Length:
50 minutes
Released:
Apr 18, 2024
Format:
Podcast episode
Description
In today's episode, Elixir Wizards Owen and Dan delve into the complexities of building advanced reporting features within software applications. They share personal insights and challenges encountered while developing reporting solutions for user-generated data, leveraging both Elixir/Phoenix and Ruby on Rails.
The discussion zeroes in on crucial data modeling and architectural decisions that enhance reporting efficiency and flexibility. Owen and Dan explore tactics like materialized views, event sourcing, and database triggers to optimize data handling while being mindful of UX elements like progress indicators and background job management.
They share insights on leveraging the Elixir/Beam ecosystem’s strengths—like concurrency and streamlined deployment—to tackle common reporting, caching, and integration challenges. The episode highlights the impact of reporting features across all aspects of a software application’s design and architecture.
Key topics discussed in this episode:
Reporting on assessment data, survey results, and user metrics
Differences between reporting and performance/error monitoring
Implementing reporting in Elixir/Phoenix vs. Ruby on Rails
Displaying reports in web, printable, PDF, SVG, and CSV formats
Challenges of generating PDFs for large data sets
Streaming CSV data directly to the client
Handling long-running report generation tasks
Providing progress indicators and user notifications
Strategies for canceling or abandoning incomplete reports
Tradeoffs of pre-calculating report data vs. real-time generation
Materializing views and denormalizing data for reporting
Exploring event sourcing patterns for reporting needs
Using database triggers and stored procedures for reporting
Balancing data structure optimization for reports vs. day-to-day usage
Caching report data for faster retrieval and rendering
Charting and visualization integration in reporting systems
Links mentioned:
Prometheus monitoring system & time series database https://prometheus.io/
Thinking Elixir "FLAME with Chris McCord" https://podcast.thinkingelixir.com/181
Phoenix LiveView Uploads https://hexdocs.pm/phoenix/fileuploads.html
https://hexdocs.pm/phoenixlive_view/Phoenix.LiveView.UploadWriter.html
Postgrex PostgreSQL driver for Elixir https://hexdocs.pm/postgrex/Postgrex.html
Ecto https://hexdocs.pm/ecto/Ecto.html
Heroku cloud application platform https://www.heroku.com/
Elixir Wizards S9E12 Marcelo Dominguez on Command and Query Responsibility Segregation https://smartlogic.io/podcast/elixir-wizards/s9-e12-marcelo-dominguez-cqrs/
Commanded Elixir CQRS/ES applications https://github.com/commanded/commanded
Tailwind CSS Framework https://github.com/tailwindlabs
Memcached https://memcached.org/
Redis https://redis.io/
Oban https://hexdocs.pm/oban/Oban.html
ETS https://hexdocs.pm/ets/ETS.html
Capistrano remote server automation and deployment tool https://capistranorb.com/
The discussion zeroes in on crucial data modeling and architectural decisions that enhance reporting efficiency and flexibility. Owen and Dan explore tactics like materialized views, event sourcing, and database triggers to optimize data handling while being mindful of UX elements like progress indicators and background job management.
They share insights on leveraging the Elixir/Beam ecosystem’s strengths—like concurrency and streamlined deployment—to tackle common reporting, caching, and integration challenges. The episode highlights the impact of reporting features across all aspects of a software application’s design and architecture.
Key topics discussed in this episode:
Reporting on assessment data, survey results, and user metrics
Differences between reporting and performance/error monitoring
Implementing reporting in Elixir/Phoenix vs. Ruby on Rails
Displaying reports in web, printable, PDF, SVG, and CSV formats
Challenges of generating PDFs for large data sets
Streaming CSV data directly to the client
Handling long-running report generation tasks
Providing progress indicators and user notifications
Strategies for canceling or abandoning incomplete reports
Tradeoffs of pre-calculating report data vs. real-time generation
Materializing views and denormalizing data for reporting
Exploring event sourcing patterns for reporting needs
Using database triggers and stored procedures for reporting
Balancing data structure optimization for reports vs. day-to-day usage
Caching report data for faster retrieval and rendering
Charting and visualization integration in reporting systems
Links mentioned:
Prometheus monitoring system & time series database https://prometheus.io/
Thinking Elixir "FLAME with Chris McCord" https://podcast.thinkingelixir.com/181
Phoenix LiveView Uploads https://hexdocs.pm/phoenix/fileuploads.html
https://hexdocs.pm/phoenixlive_view/Phoenix.LiveView.UploadWriter.html
Postgrex PostgreSQL driver for Elixir https://hexdocs.pm/postgrex/Postgrex.html
Ecto https://hexdocs.pm/ecto/Ecto.html
Heroku cloud application platform https://www.heroku.com/
Elixir Wizards S9E12 Marcelo Dominguez on Command and Query Responsibility Segregation https://smartlogic.io/podcast/elixir-wizards/s9-e12-marcelo-dominguez-cqrs/
Commanded Elixir CQRS/ES applications https://github.com/commanded/commanded
Tailwind CSS Framework https://github.com/tailwindlabs
Memcached https://memcached.org/
Redis https://redis.io/
Oban https://hexdocs.pm/oban/Oban.html
ETS https://hexdocs.pm/ets/ETS.html
Capistrano remote server automation and deployment tool https://capistranorb.com/
Released:
Apr 18, 2024
Format:
Podcast episode
Titles in the series (100)
René Föhring on Credo – Elixir Internals: Welcome back to the SmartLogic Podcast where we talk about the latest developments and best practices in the web and mobile software industry. In continuing with our theme of Elixir Internals, we’re having a conversation about the inner workings of one of the most popular Elixir libraries, Credo, and we are joined by the author René Föhring. René shares the story of how he was introduced to Elixir while doing his PhD and looking for a new programming language and then shares the philosophy and inspiration Credo was developed on. by Elixir Wizards