Nothing Special   »   [go: up one dir, main page]

Understanding the Python Traceback

Understanding the Python Traceback

by Chad Hansen basics python

Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Getting the Most Out of a Python Traceback

Python prints a traceback when an exception is raised in your code. The traceback output can be a bit overwhelming if you’re seeing it for the first time or you don’t know what it’s telling you. But the Python traceback has a wealth of information that can help you diagnose and fix the reason for the exception being raised in your code. Understanding what information a Python traceback provides is vital to becoming a better Python programmer.

By the end of this tutorial, you’ll be able to:

  • Make sense of the next traceback you see
  • Recognize some of the more common tracebacks
  • Log a traceback successfully while still handling the exception

What Is a Python Traceback?

A traceback is a report containing the function calls made in your code at a specific point. Tracebacks are known by many names, including stack trace, stack traceback, backtrace, and maybe others. In Python, the term used is traceback.

When your program results in an exception, Python will print the current traceback to help you know what went wrong. Below is an example to illustrate this situation:

Python
# example.py
def greet(someone):
    print('Hello, ' + someon)

greet('Chad')

Here, greet() gets called with the parameter someone. However, in greet(), that variable name is not used. Instead, it has been misspelled as someon in the print() call.

When you run this program, you’ll get the following traceback:

Shell
$ python example.py
Traceback (most recent call last):
  File "/path/to/example.py", line 4, in <module>
    greet('Chad')
  File "/path/to/example.py", line 2, in greet
    print('Hello, ' + someon)
NameError: name 'someon' is not defined

This traceback output has all of the information you’ll need to diagnose the issue. The final line of the traceback output tells you what type of exception was raised along with some relevant information about that exception. The previous lines of the traceback point out the code that resulted in the exception being raised.

In the above traceback, the exception was a NameError, which means that there is a reference to some name (variable, function, class) that hasn’t been defined. In this case, the name referenced is someon.

The final line in this case has enough information to help you fix the problem. Searching the code for the name someon, which is a misspelling, will point you in the right direction. Often, however, your code is a lot more complicated.

How Do You Read a Python Traceback?

The Python traceback contains a lot of helpful information when you’re trying to determine the reason for an exception being raised in your code. In this section, you’ll walk through different tracebacks in order to understand the different bits of information contained in a traceback.

Python Traceback Overview

There are several sections to every Python traceback that are important. The diagram below highlights the various parts:

An example Python traceback with call-outs.

In Python, it’s best to read the traceback from the bottom up:

  1. Blue box: The last line of the traceback is the error message line. It contains the exception name that was raised.

  2. Green box: After the exception name is the error message. This message usually contains helpful information for understanding the reason for the exception being raised.

  3. Yellow box: Further up the traceback are the various function calls moving from bottom to top, most recent to least recent. These calls are represented by two-line entries for each call. The first line of each call contains information like the file name, line number, and module name, all specifying where the code can be found.

  4. Red underline: The second line for these calls contains the actual code that was executed.

There are a few differences between traceback output when you’re executing your code in the command-line and running code in the REPL. Below is the same code from the previous section executed in a REPL and the resulting traceback output:

Python
>>> def greet(someone):
...   print('Hello, ' + someon)
... 
>>> greet('Chad')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in greet
NameError: name 'someon' is not defined

Notice that in place of file names, you get "<stdin>". This makes sense since you typed the code in through standard input. Also, the executed lines of code are not displayed in the traceback.

Specific Traceback Walkthrough

Going through some specific traceback output will help you better understand and see what information the traceback will give you.

The code below is used in the examples following to illustrate the information a Python traceback gives you:

Python
# greetings.py
def who_to_greet(person):
    return person if person else input('Greet who? ')

def greet(someone, greeting='Hello'):
    print(greeting + ', ' + who_to_greet(someone))

def greet_many(people):
    for person in people:
        try:
            greet(person)
        except Exception:
            print('hi, ' + person)

Here, who_to_greet() takes a value, person, and either returns it or prompts for a value to return instead.

Then, greet() takes a name to be greeted, someone, and an optional greeting value and calls print(). who_to_greet() is also called with the someone value passed in.

Finally, greet_many() will iterate over the list of people and call greet(). If there is an exception raised by calling greet(), then a simple backup greeting is printed.

This code doesn’t have any bugs that would result in an exception being raised as long as the right input is provided.

If you add a call to greet() to the bottom of greetings.py and specify a keyword argument that it isn’t expecting (for example greet('Chad', greting='Yo')), then you’ll get the following traceback:

Shell
$ python example.py
Traceback (most recent call last):
  File "/path/to/greetings.py", line 19, in <module>
    greet('Chad', greting='Yo')
TypeError: greet() got an unexpected keyword argument 'greting'

Once again, with a Python traceback, it’s best to work backward, moving up the output. Starting at the final line of the traceback, you can see that the exception was a TypeError. The messages that follow the exception type, everything after the colon, give you some great information. It tells you that greet() was called with a keyword argument that it didn’t expect. The unknown argument name is also given to you: greting.

Moving up, you can see the line that resulted in the exception. In this case, it’s the greet() call that we added to the bottom of greetings.py.

The next line up gives you the path to the file where the code exists, the line number of that file where the code can be found, and which module it’s in. In this case, because our code isn’t using any other Python modules, we just see <module> here, meaning that this is the file that is being executed.

With a different file and different input, you can see the traceback really pointing you in the right direction to find the issue. If you are following along, remove the buggy greet() call from the bottom of greetings.py and add the following file to your directory:

Python
# example.py
from greetings import greet

greet(1)

Here you’ve set up another Python file that is importing your previous module, greetings.py, and using greet() from it. Here’s what happens if you now run example.py:

Shell
$ python example.py
Traceback (most recent call last):
  File "/path/to/example.py", line 3, in <module>
    greet(1)
  File "/path/to/greetings.py", line 5, in greet
    print(greeting + ', ' + who_to_greet(someone))
TypeError: must be str, not int

The exception raised in this case is a TypeError again, but this time the message is a little less helpful. It tells you that somewhere in the code it was expecting to work with a string, but an integer was given.

Moving up, you see the line of code that was executed. Then the file and line number of the code. This time, however, instead of <module>, we get the name of the function that was being executed, greet().

Moving up to the next executed line of code, we see our problematic greet() call passing in an integer.

Sometimes after an exception is raised, another bit of code catches that exception and also results in an exception. In these situations, Python will output all exception tracebacks in the order in which they were received, once again ending in the most recently raise exception’s traceback.

Since this can be a little confusing, here’s an example. Add a call to greet_many() to the bottom of greetings.py:

Python
# greetings.py
...
greet_many(['Chad', 'Dan', 1])

This should result in printing greetings to all three people. However, if you run this code, you’ll see an example of the multiple tracebacks being output:

Shell
$ python greetings.py
Hello, Chad
Hello, Dan
Traceback (most recent call last):
  File "greetings.py", line 10, in greet_many
    greet(person)
  File "greetings.py", line 5, in greet
    print(greeting + ', ' + who_to_greet(someone))
TypeError: must be str, not int

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "greetings.py", line 14, in <module>
    greet_many(['Chad', 'Dan', 1])
  File "greetings.py", line 12, in greet_many
    print('hi, ' + person)
TypeError: must be str, not int

Notice the highlighted line starting with During handling in the output above. In between all tracebacks, you’ll see this line. Its message is very clear, while your code was trying to handle the previous exception, another exception was raised.

You have seen the previous exception before, when you called greet() with an integer. Since we added a 1 to the list of people to greet, we can expect the same result. However, the function greet_many() wraps the greet() call in a try and except block. Just in case greet() results in an exception being raised, greet_many() wants to print a default greeting.

The relevant portion of greetings.py is repeated here:

Python
def greet_many(people):
    for person in people:
        try:
            greet(person)
        except Exception:
            print('hi, ' + person)

So when greet() results in the TypeError because of the bad integer input, greet_many() handles that exception and attempts to print a simple greeting. Here the code ends up resulting in another, similar, exception. It’s still attempting to add a string and an integer.

Seeing all of the traceback output can help you see what might be the real cause of an exception. Sometimes when you see the final exception raised, and its resulting traceback, you still can’t see what’s wrong. In those cases, moving up to the previous exceptions usually gives you a better idea of the root cause.

What Are Some Common Tracebacks in Python?

Knowing how to read a Python traceback when your program raises an exception can be very helpful when you’re programming, but knowing some of the more common tracebacks can also speed up your process.

Here are some common exceptions you might come across, the reasons they get raised and what they mean, and the information you can find in their tracebacks.

AttributeError

The AttributeError is raised when you try to access an attribute on an object that doesn’t have that attribute defined. The Python documentation defines when this exception is raised:

Raised when an attribute reference or assignment fails. (Source)

Here’s an example of the AttributeError being raised:

Python
>>> an_int = 1
>>> an_int.an_attribute
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute 'an_attribute'

The error message line for an AttributeError tells you that the specific object type, int in this case, doesn’t have the attribute accessed, an_attribute in this case. Seeing the AttributeError in the error message line can help you quickly identify which attribute you attempted to access and where to go to fix it.

Most of the time, getting this exception indicates that you are probably working with an object that isn’t the type you were expecting:

Python
>>> a_list = (1, 2)
>>> a_list.append(3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'tuple' object has no attribute 'append'

In the example above, you might be expecting a_list to be of type list, which has a method called .append(). When you receive the AttributeError exception and see that it was raised when you are trying to call .append(), that tells you that you probably aren’t dealing with the type of object you were expecting.

Often, this happens when you are expecting an object to be returned from a function or method call to be of a specific type, and you end up with an object of type None. In this case, the error message line will read, AttributeError: 'NoneType' object has no attribute 'append'.

ImportError

The ImportError is raised when something goes wrong with an import statement. You’ll get this exception, or its subclass ModuleNotFoundError, if the module you are trying to import can’t be found or if you try to import something from a module that doesn’t exist in the module. The Python documentation defines when this exception is raised:

Raised when the import statement has troubles trying to load a module. Also raised when the ‘from list’ in from ... import has a name that cannot be found. (Source)

Here’s an example of the ImportError and ModuleNotFoundError being raised:

Python
>>> import asdf
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'asdf'
>>> from collections import asdf
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'asdf'

In the example above, you can see that attempting to import a module that doesn’t exist, asdf, results in the ModuleNotFoundError. When attempting to import something that doesn’t exist, asdf, from a module that does exists, collections, this results in an ImportError. The error message lines at the bottom of the tracebacks tell you which thing couldn’t be imported, asdf in both cases.

IndexError

The IndexError is raised when you attempt to retrieve an index from a sequence, like a list or a tuple, and the index isn’t found in the sequence. The Python documentation defines when this exception is raised:

Raised when a sequence subscript is out of range. (Source)

Here’s an example that raises the IndexError:

Python
>>> a_list = ['a', 'b']
>>> a_list[3]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range

The error message line for an IndexError doesn’t give you great information. You can see that you have a sequence reference that is out of range and what the type of the sequence is, a list in this case. That information, combined with the rest of the traceback, is usually enough to help you quickly identify how to fix the issue.

KeyError

Similar to the IndexError, the KeyError is raised when you attempt to access a key that isn’t in the mapping, usually a dict. Think of this as the IndexError but for dictionaries. The Python documentation defines when this exception is raised:

Raised when a mapping (dictionary) key is not found in the set of existing keys. (Source)

Here’s an example of the KeyError being raised:

Python
>>> a_dict['b']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'b'

The error message line for a KeyError gives you the key that could not be found. This isn’t much to go on but, combined with the rest of the traceback, is usually enough to fix the issue.

For an in-depth look at KeyError, take a look at Python KeyError Exceptions and How to Handle Them.

NameError

The NameError is raised when you have referenced a variable, module, class, function, or some other name that hasn’t been defined in your code. The Python documentation defines when this exception is raised:

Raised when a local or global name is not found. (Source)

In the code below, greet() takes a parameter person. But in the function itself, that parameter has been misspelled to persn:

Python
>>> def greet(person):
...     print(f'Hello, {persn}')
>>> greet('World')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in greet
NameError: name 'persn' is not defined

The error message line of the NameError traceback gives you the name that is missing. In the example above, it’s a misspelled variable or parameter to the function that was passed in.

A NameError will also be raised if it’s the parameter that you misspelled:

Python
>>> def greet(persn):
...     print(f'Hello, {person}')
>>> greet('World')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in greet
NameError: name 'person' is not defined

Here, it might seem as though you’ve done nothing wrong. The last line that was executed and referenced in the traceback looks good. If you find yourself in this situation, then the thing to do is to look through your code for where the person variable is used and defined. Here you can quickly see that the parameter name was misspelled.

SyntaxError

The SyntaxError is raised when you have incorrect Python syntax in your code. The Python documentation defines when this exception is raised:

Raised when the parser encounters a syntax error. (Source)

Below, the problem is a missing colon that should be at the end of the function definition line. In the Python REPL, this syntax error is raised right away after hitting enter:

Python
>>> def greet(person)
  File "<stdin>", line 1
    def greet(person)
                    ^
SyntaxError: invalid syntax

The error message line of the SyntaxError only tells you that there was a problem with the syntax of your code. Looking into the lines above gives you the line with the problem and usually a ^ (caret) pointing to the problem spot. Here, the colon is missing from the function’s def statement.

Also, with SyntaxError tracebacks, the regular first line Traceback (most recent call last): is missing. That is because the SyntaxError is raised when Python attempts to parse your code, and the lines aren’t actually being executed.

TypeError

The TypeError is raised when your code attempts to do something with an object that can’t do that thing, such as trying to add a string to an integer or calling len() on an object where its length isn’t defined. The Python documentation defines when this exception is raised:

Raised when an operation or function is applied to an object of inappropriate type. (Source)

Following are several examples of the TypeError being raised:

Python
>>> 1 + '1'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'int' and 'str'
>>> '1' + 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: must be str, not int
>>> len(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: object of type 'int' has no len()

All of the above examples of raising a TypeError results in an error message line with different messages. Each of them does a pretty good job of informing you of what is wrong.

The first two examples attempt to add strings and integers together. However, they are subtly different:

  • The first is trying to add a str to an int.
  • The second is trying to add an int to a str.

The error message lines reflect these differences.

The last example attempts to call len() on an int. The error message line tells you that you can’t do that with an int.

ValueError

The ValueError is raised when the value of the object isn’t correct. You can think of this as an IndexError that is raised because the value of the index isn’t in the range of the sequence, only the ValueError is for a more generic case. The Python documentation defines when this exception is raised:

Raised when an operation or function receives an argument that has the right type but an inappropriate value, and the situation is not described by a more precise exception such as IndexError. (Source)

Here are two examples of ValueError being raised:

Python
>>> a, b, c = [1, 2]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: not enough values to unpack (expected 3, got 2)
>>> a, b = [1, 2, 3]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: too many values to unpack (expected 2)

The ValueError error message line in these examples tells you exactly what the problem is with the values:

  1. In the first example, you are trying to unpack too many values. The error message line even tells you that you were expecting to unpack 3 values but got 2 values.

  2. In the second example, the problem is that you are getting too many values and not enough variables to unpack them into.

How Do You Log a Traceback?

Getting an exception and its resulting Python traceback means you need to decide what to do about it. Usually fixing your code is the first step, but sometimes the problem is with unexpected or incorrect input. While it’s good to provide for those situations in your code, sometimes it also makes sense to silence or hide the exception by logging the traceback and doing something else.

Here’s a more real-world example of code that needs to silence some Python tracebacks. This example uses the requests library. You can find out more about it in Python’s Requests Library (Guide):

Python
# urlcaller.py
import sys
import requests

response = requests.get(sys.argv[1])

print(response.status_code, response.content)

This code works well. When you run this script, giving it a URL as a command-line argument, it will call the URL and then print the HTTP status code and the content from the response. It even works if the response was an HTTP error status:

Shell
$ python urlcaller.py https://httpbin.org/status/200
200 b''
$ python urlcaller.py https://httpbin.org/status/500
500 b''

However, sometimes the URL your script is given to retrieve doesn’t exist, or the host server is down. In those cases, this script will now raise an uncaught ConnectionError exception and print a traceback:

Python Traceback
$ python urlcaller.py http://thisurlprobablydoesntexist.com
...
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "urlcaller.py", line 5, in <module>
    response = requests.get(sys.argv[1])
  File "/path/to/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/path/to/requests/api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "/path/to/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/path/to/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/path/to/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='thisurlprobablydoesntexist.com', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7faf9d671860>: Failed to establish a new connection: [Errno -2] Name or service not known',))

The Python traceback here can be very long with many other exceptions being raised and finally resulting in the ConnectionError being raised by requests itself. If you move up the final exceptions traceback, you can see that the problem all started in our code with line 5 of urlcaller.py.

If you wrap the offending line in a try and except block, catching the appropriate exception will allow your script to continue to work with more inputs:

Python
# urlcaller.py
...
try:
    response = requests.get(sys.argv[1])
except requests.exceptions.ConnectionError:
    print(-1, 'Connection Error')
else:
    print(response.status_code, response.content)

The code above uses an else clause with the try and except block. If you’re unfamiliar with this feature of Python, then check out the section on the else clause in Python Exceptions: An Introduction.

Now when you run the script with a URL that will result in a ConnectionError being raised, you’ll get printed a -1 for the status code, and the content Connection Error:

Shell
$ python urlcaller.py http://thisurlprobablydoesntexist.com
-1 Connection Error

This works great. However, in most real systems, you don’t want to just silence the exception and resulting traceback, but you want to log the traceback. Logging tracebacks allows you to have a better understanding of what goes wrong in your programs.

You can log the traceback in the script by importing the logging package, getting a logger, and calling .exception() on that logger in the except portion of the try and except block. Your final script should look something like the following code:

Python
# urlcaller.py
import logging
import sys
import requests

logger = logging.getLogger(__name__)

try:
    response = requests.get(sys.argv[1])
except requests.exceptions.ConnectionError as e:
    logger.exception()
    print(-1, 'Connection Error')
else:
    print(response.status_code, response.content)

Now when you run the script for a problematic URL, it will print the expected -1 and Connection Error, but it will also log the traceback:

Python Traceback
$ python urlcaller.py http://thisurlprobablydoesntexist.com
...
  File "/path/to/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='thisurlprobablydoesntexist.com', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7faf9d671860>: Failed to establish a new connection: [Errno -2] Name or service not known',))
-1 Connection Error

By default, Python will send log messages to standard error (stderr). This looks like we haven’t suppressed the traceback output at all. However, if you call it again while redirecting the stderr, you can see that the logging system is working, and we can save our logs off for later:

Shell
$ python urlcaller.py http://thisurlprobablydoesntexist.com 2> my-logs.log
-1 Connection Error

Conclusion

The Python traceback contains great information that can help you find what is going wrong in your Python code. These tracebacks can look a little intimidating, but once you break it down to see what it’s trying to show you, they can be super helpful. Going through a few tracebacks line by line will give you a better understanding of the information they contain and help you get the most out of them.

Getting a Python traceback output when you run your code is an opportunity to improve your code. It’s one way Python tries to help you out.

Now that you know how to read a Python traceback, you can benefit from learning more about some tools and techniques for diagnosing the problems that your traceback output is telling you about. Python’s built-in traceback module can be used to work with and inspect tracebacks. The traceback module can be helpful when you need to get more out of the traceback output. It would also be helpful to learn more about some techniques for debugging your Python code and ways to debug in IDLE.

Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Getting the Most Out of a Python Traceback

🐍 Python Tricks 💌

Get a short & sweet Python Trick delivered to your inbox every couple of days. No spam ever. Unsubscribe any time. Curated by the Real Python team.

Python Tricks Dictionary Merge

About Chad Hansen

Chad is an avid Pythonista and does web development with Django fulltime. Chad lives in Utah with his wife and six kids.

» More about Chad

Each tutorial at Real Python is created by a team of developers so that it meets our high quality standards. The team members who worked on this tutorial are:

Master Real-World Python Skills With Unlimited Access to Real Python

Locked learning resources

Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas:

Level Up Your Python Skills »

Master Real-World Python Skills
With Unlimited Access to Real Python

Locked learning resources

Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas:

Level Up Your Python Skills »

What Do You Think?

Rate this article:

What’s your #1 takeaway or favorite thing you learned? How are you going to put your newfound skills to use? Leave a comment below and let us know.

Commenting Tips: The most useful comments are those written with the goal of learning from or helping out other students. Get tips for asking good questions and get answers to common questions in our support portal.


Looking for a real-time conversation? Visit the Real Python Community Chat or join the next “Office Hours” Live Q&A Session. Happy Pythoning!