Nothing Special   »   [go: up one dir, main page]

0% found this document useful (0 votes)
10 views32 pages

Sentry vs crashanalytics

Download as pdf or txt
Download as pdf or txt
Download as pdf or txt
You are on page 1/ 32

Sentry vs.

Crashlytics:
The Mobile Developer’s
Decision-Making Guide
Contents
Introduction 1

At-a-glance overview 2

Features comparison

Context and insight 5

Workflow 13

Innovation and Scale 20

Assessing ROI 26

Getting started with Sentry 28

Conclusion 30
Introduction
Many mobile developers weigh the differences between Firebase
Crashlytics and Sentry when they are trying to choose the right
crash, error, and performance monitoring solution for their apps.
This guide, which includes a feature comparison, tools to assess ROI,
customer stories, and more, will help you make an informed decision.

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Introduction / 1


At-a-glance overview
Firebase Crashlytics is a lightweight crash In this guide, we will focus on the “As we grow and move
reporting tool. Part of the Firebase app crash reporting and error monitoring
upmarket, we have tighter
development platform from Google, capabilities of Crashlytics and Sentry.
it offers support for Android, iOS, Both Crashlytics and Sentry synthesize requirements around reliability
Flutter, and Unity. Firebase also offers crashes and errors into manageable lists and issue management.
Performance Monitoring, which collects of issues, offering contextual information
Reporting through Crashlytics
and analyzes performance characteristics to help you find and fix root causes.
of Apple, Android, and web apps. However, the two offerings vary in their wasn’t working well. We
levels of customization, product depth,
wanted to up our game for
Sentry is a complete mobile monitoring and flexibility.
solution, including crash reporting and mobile. We wanted to capture
error and performance monitoring. In general, Firebase Crashlytics is a fine non-fatal errors and triage
Part of a platform that also provides solution for hobbyists, solopreneurs,
them so we could respond to
monitoring for web apps and back end and small teams. Sentry is a better
projects, Sentry mobile monitoring offers solution for growing teams that need them faster.”
out-of-the-box support for Android, iOS, deeper context and insights into solving
Flutter, Unity, React Native, and more. crashes and errors, customizable
Staff Engineer,
workflow tools, and product investments
Health Benefits Platform
to enable innovation and scale.

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / At-a-glance overview / 2
Here’s a high-level overview of the differences
between Firebase Crashlytics and Sentry.

Sentry Firebase Crashlytics


Offers metadata, stack traces, breadcrumbs, suspect
Offers metadata, stack traces, keys, logs
commits, screenshots, view hierarchy

Context Source Map support for React Native No Source Map support for React Native
and Insights
Search and filter by custom tags; view trends and Limited search and filter capabilities; export and
analyze data with Dashboards and Discover analyze data in BigQuery

Fully customizable alerts Pre-set alerts

Workflow Automatic issue assignment and notifications No issue assignment or targeted notifications

Two-way data flow between other tools One-way data flow to other tools

Product investments focused on enabling developers Product investments focused on deeper integration
to adopt new technologies with Firebase Platform
Innovation
Vendor agnostic Reliant on Firebase Platform
and Scale
EU Data residency Not GDPR compliant

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / At-a-glance overview / 3
Features comparison
Let’s take a deeper dive into these 3 categories.

• Context and Insights


• Workflow
• Innovation and scale

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Features comparison / 4
Context and Insights
Prioritizing crashes and errors and identifying root causes
in order to solve them fast is the primary use case for crash
reporting tools. So let’s start there.

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Context and Insights / 5
Here’s a look at the Crashlytics dashboard,
where you can view crash-free rate
statistics, event trends, and issues for one
project at a time.

To get more granular details, you can filter


crash-free stats and event trends by release
version or event type. You can also drill down
into specific issues and filter by issue state,
device, operating system, or custom keys.
Additionally, you can search issues by title or
custom keys.
Filter crash-free rates: Filter issues:

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Context and Insights / 6
Let’s compare Sentry, starting with the On the Project Details page, you can see We have only just started the guide,
Issues page. Here, you can search and current crash free sessions and Apdex but you likely already have a sense for the
filter issues across projects and perform scores for specific release versions, build, deeper functionality available with Sentry.
advanced searches with default and package, or stage of usage. You can also Let’s move on to context.
custom tokens. visualize crash free sessions over time and
see any recent alerts.

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Context and Insights / 7
Once you have identified a “There’s not a lot of depth
crash or non-fatal error to solve,
to the data that Crashlytics
Crashlytics offers stack traces,
collects. There’s not a lot of
keys, logs, and data. We hear from
a lot of users that the content isn’t breadcrumbing, there’s not a
usually enough to solve most bugs. lot of ability to dig in deeper
other than one line of stack
trace causing an issue”

Head of Engineering,
Music Streaming Service
Debugging with Crashlytics is especially
difficult if you build with React Native.
Crashlytics does not provide source map
support for React native out-of-the-box,
which results in minified stack traces that
are difficult to read.

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Context and Insights / 8
Sentry, on the other hand, provides deep actionable context to help you
solve issues faster.

• Suspect Commits to uncover the • Gain insight into the sequence of events
commits that introduced the error that led to the error, as well as the line of
code that contains the bug, with Stack
• Tags to index and search key information Traces
like device type or OS
• If the cause of an error isn’t obvious,
• Screenshots to see what your users saw trace your user’s steps with
when the app crashed Breadcrumbs

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Context and Insights / 9
In fact, in the recent State of “We had been using Firebase
React Native survey, Sentry was
Crashlytics but it does not support
identified as the most popular
source maps from React Native,
crash reporting solution among
survey respondents. so it was always difficult trying to
understand exactly where a crash
originated from and what change
might’ve introduced it. Sentry makes
it easy to identify new crashes and
where they are coming from.”
For React Native users, Sentry
provides unminified stack traces.
Frikkie Snyman,
Mobile Platform Engineer, Relive

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Context and Insights / 10
Lastly, let’s take a look at the comparative insights and analytics
offerings from Crashlytics and Sentry.

If you are using BigQuery


streaming, which adds
additional costs, data is
shown in real-time if you
refresh the page.

BigQuery gives you access to your raw Crashlytics provides a Data Studio
Crashlytics data through an integration so template, which is powered by exported
you can get more in-depth crash reporting crash data.
and run custom queries. You can set up
ways to automate your release process.

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Context and Insights / 11
Sentry provides two products that offer Sentry’s Dashboards provide you with The Releases page provides a visualization
analytics tools built right into the platform: a broad overview of your application’s of your releases. It presents adoption
Discovery and Dashboards health by allowing you to navigate through of releases from the past 24 hours and
crash and error across multiple projects. provides a high-level view of:
Discover provides visibility into your data Dashboards are made up of one or more
across environments by building upon widgets, and each widget visualizes one or
• Each release version

and enriching your error data. You can more Discover queries. • The associated project

query and unlock insights into the health • The adoption stage of each release
of your entire system and get answers You can build custom queries in Discover • The authors of each commit
to critical business questions — all in one and custom Dashboards on Sentry’s • The percentage of crash-free users
place. Use Discover to view comprehensive Business plan. • The percentage of crash-free sessions
information sent to Sentry.

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Context and Insights / 12
Workflow
Workflows are an area where you will see clear
advantages to Sentry over Crashlytics, especially for
growing teams that rely on collaboration to ship and
improve their apps. Most workflows to solve crashes
start by finding out that there is a problem. So let’s
take a look at alerts and notifications.

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Workflow / 13


Crashlytics offers the following
pre-set alerts:

• New issue alerts


• Regressing detection alerts
• Trending issue digests
• Velocity alerts

You can adjust the Velocity alert settings.

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Workflow / 14


By default, Firebase will send Crashlytics “There’s a lot of noise [with
alerts via email to every project member.
Crashlytics] and it’s not useful

For velocity alerts and regressions, noise. We’ll get alerts all hours
Firebase will show these alerts in the of the night. I’ll get alerts that
Firebase console. Firebase can also send something’s going wrong and it
one-way summaries to Slack, Jira, or
ends up being one user causing
PagerDuty.
a lot of noise. And so the alerting
On the paid Blaze plan for Firebase, you can I find to not be super helpful.
use Cloud Functions to send customized
Alerts should be intelligent.
notifications for velocity alerts to additional
And when you’re saying that
third-party systems.
there’s a critical issue and it
impacts five users... not worth
With Crashlytics, you cannot assign issues
to a specific team or developer. As such,
waking the team up for.”
you cannot send notifications only to an
assigned person or team. One developer’s Head of Engineering, Music Streaming
bug becomes everyone’s problem. Service

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Workflow / 15


Crashlytics notifications to Slack and Jira Similarly, you can create tickets in Jira
are “one-way”, meaning that updates with Crashlytics, but any updates made in
don’t sync between tools. If you receive a Jira will not be automatically reflected in
notification in Slack, you then must log back Crashlytics.
into Crashlytics to take action.

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Workflow / 16


Let’s take a look now at the ability to Sentry offers fully customizable alerts that With Ownership Rules, you can
customize your workflow with Sentry, which allow you to fine-tune notifications and automatically assign issues and send alerts
can help developers stay focused only on reduce noise. to the specific team or developer that is
crashes and bugs that need their attention. best positioned to solve the issue fast.
You can create two types of alerts: With Code Owners, you can import GitHub
or GitLab CODEOWNERS and automatically
• Issue alerts that trigger when an issue assign issues according to those file paths.
matches a specific criteria

• Metric alerts when macro-level metrics


cross specific thresholds

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Workflow / 17


You can choose to notify specific teams or With our Jira integration, Sentry will automatically
developers via email, Slack, multiple supported create Jira tickets. Any updates made to the Jira
integrations (Opsgenie, PagerDuty, Discord, ticket, like delegating the issue to an assignee or
Microsoft Teams, Jira, and more), and custom updating a status on Jira, will also populate in Sentry.
integrations through webhooks.

Sentry offers two-way integrations with many


communication and issues tracking tools. For
example, you can triage, resolve, and archive Sentry
issues directly in Slack, without switching back to
Sentry.

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Workflow / 18


If you are building a mobile app solo or “Our workflow has changed from being reactive to proactive, with the
with a very small team, you may not notice
ability for each team to monitor issues over time, customize alerts
minor snags in your workflow. But, as your
based on what’s important for their product area.”
app grows and multiple developers are
collaborating across teams, inefficiencies in
your workflow can really slow you down. Senior Head of Engineering, Global Food Delivery Service

Sentry’s workflow tools can streamline


processes for your team, helping them
identify and solve crashes faster, allowing
them to deliver more stable apps and focus
more time on building more features for
their customers.

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Workflow / 19


Innovation and Scale
Let’s move on to the final part of our product
Crashlytics vs Sentry comparison: innovation and
scale. Here we will start with Sentry.

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Innovation and Scale / 20
Since launching our first mobile SDKs for Android and iOS in 2016, Sentry has continued to invest in our mobile offerings. We have especially
doubled down in the last three years as mobile adoption has continued to accelerate after the pandemic.

Here is a look at Sentry’s recent crash reporting investments, which are focused on helping developers solve crashes and non-fatal errors
faster and adopt new technologies.

2020 2021 2022 2023


• Release Health • Mobile Screenshots • Dexguard Support • View Hierarchy

• Semantic Versioning
• Android SDK • Android SDK v6.0
• Device Classification
• Auto activity instrumentation • Jetpack compose support

• Android Symbol Collection • Cocoa SDK v7.0


• Cocoa SDK • Android SDK v7.0
• iOS App Hangs ANR improvements

• Andrioid SDK v3.0 • React Native SDK v3.0


• React Native • Cocoa SDK v8.0
Attachment Support • SwiftUI support

• Cocoa SDK v6.0 • Flutter SDK v6.0


• Flutter SDK • React Native SDK 5.0
• Mobile Vitals New architecture support

• React Native SDK v2.0 • Unity SDK launched


• Flutter SDK v7.0
• Screenshots

• Flutter SDK launched • Unity SDK v1.0

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Innovation and Scale / 21
“Sentry’s team is often working on
something interesting, at a super fast
cadence. Other tools I’ve used before
just can’t compare in that regard.
We’re keeping an eye on their Kotlin
Multiplatform SDK and we might adopt it
eventually. ”

Fred Porciuncula, Mobile Engineer, Planet Wild

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Innovation and Scale / 22
Let’s take a look at the Crashlytics product investment areas. A quick reminder of their
history helps inform their investment approach.

Launched Acquired by Twitter, Ranked #1 most Acquired from Fabric deprecated,


part of Fabric adopted crash Twitter by Google Crashlytics available
reporting SDK through Firebase only

2011 2013 2015 2017 2020 2023

Crashlytics was first launched in 2011. Two years later, in 2013, it was acquired by Twitter,
and integrated with the Fabric developer platform. In 2017, Google acquired Crashlytics and
then focused development over the next three years on integrating Crashlytics with the
Firebase Platform, eventually deprecating Fabric and the previous version of Crashlytics.

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Innovation and Scale / 23
Many of the major updates to Crashlytics are around a deeper integration with the Firebase ecosystem, rather than investments in new
technologies in mobile app development (Jetpack Compose support, for example).

2020 2021 2022 2023


Firebase ecosystem Firebase ecosystem Firebase ecosystem
• Firebase Crashlytics SDK (removing • Crashes correctly attributed to .kt • Firebase Crashlytics insights in Navigate from stack trace to code in
Fabric dependencies) files Android Studio Android Studio

Other updates Other updates


• Bulk issues management • Custom alerting with Cloud • Filter Android app events by device
Functions for Firebase category

• Crashlytics Signals
• Updated Flutter plugin to track
• GWP-ASan reporting
fatal errors on demand

• Filter crashes by Google Play track • Crash grouping with variants

• dSYM file “Drag and Drop”

• Moved event type into new


metadata line

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Innovation and Scale / 24
On a related note – it’s important to consider the pros and cons of relying on a crash reporting tool that is so deeply embedded within the
Firebase Platform. Although the Firebase Platform can simplify many aspects of mobile application development, it has its downsides and
limitations, including:

CRASHLYTICS SENTRY
Conversely, Sentry offers the following benefits:

• Vendor agnostic: Sentry is vendor-agnostic and supports


• Vendor lock-in: Migrating off platform may be challenging. various platform integrations as a first-class citizen. If you
decide to use non-Google products (e.g. Postgres or Mongo
instead of BigQuery for database querying), Sentry can
integrate easily while Firebase cannot.
• Cost: While Firebase offers a free tier, it can become expensive
as your app scales. You may find yourself incurring costs for
services like Cloud Firestore, Realtime Database, or Firebase • Cost: Sentry offers flexible and transparent user-based pricing.
Hosting.
• Limited investment in iOS and React Native: While advertised
as a cross-platform solution, Firebase focuses more on Android • Equal investment across multiple mobile SDKs including
and Flutter (both part of the Google ecosystem) instead of iOS Android, iOS, React Native, Flutter, Unity, and others.
and React Native.
• Data ownership and privacy: Storing sensitive or critical data
in Firebase means it’s hosted on Google’s servers. Depending
• Data ownership and privacy: As part of GDPR requirement for
on your app’s use case, this could raise concerns about data data residency, you can opt into having Sentry host your data in
ownership and privacy. the EU.

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Innovation and Scale / 25
Assessing ROI
We understand that for many users Firebase Crashlytics
may seem like the more economical choice, especially
if you already use many parts of the Firebase Platform
and Crashlytics is included at no cost. When assessing
the true cost of a solution, it’s important to assess the
financial impact of not solving crashes fast. You can try
Sentry free and see for yourself.

There are a few variables to consider:

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Assessing ROI / 26
Customer Acquisition Costs Customer churn costs Developer productivity costs
Given the competitive landscape, especially for You also need to take into account losses in One last lens to consider when evaluating the
consumer apps, it is expensive to differentiate terms of customer value. ROI of your crash reporting tool is the cost to
and market a mobile app. You need to develop developer productivity.
and pay for ad placement to target your primary Given the competitive landscape, especially for
audience segments. consumer apps, it is expensive to differentiate Although developer productivity cannot depend
and market a mobile app. You need to develop on a single metric, one important metric to
On average, it costs $75 to acquire a paying and pay for ad placement to target your primary consider when it comes to crash reporting is
mobile user. audience segments. MTTR or Mean Time to Repair. It measures the
time from when the incident is first identified to
Consider that 62% of users will uninstall an app if On average, it costs $75 to acquire a paying the time it is fixed. You can calculate MTTR by
they experience crashes, freezes, or other major mobile user. adding up the total time spent on repairs during
errors. any given period and then dividing that time by
the number of repairs.
Consider that 62% of users will uninstall an app if
If you and your team are unable to solve a crash they experience crashes, freezes, or other major
fast, that crash could cost up to $4,650 for errors. Without proper alerting, assignment, and
every 100 users of your app, just in terms of debugging tools, MTTR can be very high. If your
acquisition costs team needs to solve one crash every week, and
If you and your team are unable to solve a crash
they spend one hour longer solving that crash
fast, that crash could cost up to $4,650 for
with Crashlytics than they would with Sentry,
every 100 users of your app, just in terms of
that is more than a full week of developer time
acquisition costs
spent on fixing crashes alone. That is time that
could be spent focused on further developing
the feature set of your app.

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Assessing ROI / 27
Getting started
with Sentry
If you would like to see for yourself how Sentry
can help your team build crash free apps with
less developer time, it’s easy to get started – for
most mobile SDKs, you just need to run one line
of code.

Get Started for free,

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Getting started with Sentry / 28
Android: brew install getsentry/tools/sentry-wizard && sentry-wizard -i android

iOS: brew install getsentry/tools/sentry-wizard && sentry-wizard -i ios

Add the @sentry/react-native dependency:


npx @sentry/wizard@latest -s -i reactNative
React Native: Wrap your root component
export default Sentry.wrap(App);

Get the SDK from pub.dev by adding the following to your pubspec.yaml
dependencies:
sentry _ flutter: ^7.10.1

Import Sentry and initialize it:


import 'package:flutter/widgets.dart';
import 'package:sentry _ flutter/sentry _ flutter.dart';
Flutter
Future<void> main() async {
await SentryFlutter.init(
(options) => options.dsn = 'https://<key>@sentry.io/<project>',
appRunner: () => runApp(MyApp()),
);
// or define SENTRY _ DSN via Dart environment variable (--dart-define)
}

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Getting started with Sentry / 29
In conclusion
Now that you have a firm understanding To learn more:
of the differences between Sentry and
Crashlytics, you can make an informed Join our Discord
decision. If you are building a prototype app
solo or with a small team and are deeply Check out GitHub
invested in the Firebase Platform, then
Crashlytics may be a fine choice. Read our docs

If you and your teammates are committed Still have questions about how
to building crash-free apps in less time, Sentry can uplevel your mobile crash
then Sentry is an easy choice. With Sentry, reporting? Talk to a Sentry expert to
you can find and fix errors fast with request a demo.
deeper context and insights into solving
crashes and errors, customizable workflow
tools, and product investments to enable
innovation and scale.

Sentry vs. Crashlytics: The Mobile Developer’s Decision-Making Guide / Conclusion / 30

You might also like