Accessing Nested Files in our Python App
Python by default cannot find a nested file.
When our Python project starts to get more complex we will inevetably write multiple .py files and then we will start to organize our .py files into directories and sub-directories.
Now Python has an
import statement which allows us to reference a module (a .py file) from another module. The
import statement will allow us to bring a variable, a function, a class or the total contents of a file into another file. Below is a table of contents for this tutorial. So if you are just skimming for an answer to 'nested files' please just jump down to the __init__.py section.
Table of Contents:
- A look at our file structure
- Code in one
- Code in both
And as a side note I really like the syntax and readability of Pythons implementation of requiring files/modules. I believe it is one of the better implementations in the industry.
Common ways you will see the
import statement used:
# bring the whole object import os # only bring a specific item from time import timezone # we can rename or 'alias' our import from myFile import function as fn
So I mentioned above in the sub-title that by default Pythons
import statement cannot find a nested file. But before we get to how to make Python find our files lets look at what Python can recognize as a default.
- Standard Library Modules
- Any file that is on the same "level" as it is.
The standard library is all the prewritten code that comes with the stock Python language. These are items like math functionality, operating system functions, time and date methods, etc. Most all the common operations that you would expect a programming language to have. The second item is just saying that whatever is on the same 'level' (read here not nested) as the file being called.
2. A look at our file structure
Our file structure for this tutorial will look like this.
/myApp |- main.py | |- myFile.py | |- /Dir1 |- file1.py | |- /Dir2 |- file2.py * NOTE: a forward slash '/' indicates a directory
If we run
python main.py from our /myApp directory
# main.py import sys # ok -> this is part of standard library from time import timezone # ok -> this is part of standard library import myFile # ok -> this is on the same 'level' import file1 # fail -> ModuleNotFoundError: No module named 'file1'
The first three lines in main.py will execute fine but the fourth line will throw an error because Python only knows to look in the two places we spoke about above.
So before we go into the solution lets define some terms:
Blocks: a piece of Python code that will be executed as a unit. An Example is: a module, a function body and class definitions. Please note here that a module can be a .py file that we define.
Module: An organizational unit of Python code. Modules have a namespace and are loaded by Python via importing.
Package: A module with a __path__ attribute and can contain submodules or subpackages.
Python has two types of packages, regular packages and namespace packages. So a regular package is created by putting a
__init__.py file in a directory. And when the regular package is imported into a file the
__init__.py is executed. But even though this
__init__.py file is executed one thing that is sort of strange is that this file can be left empty of contents (an empty file).
__init__.py file will require that the files listed in it are loaded in the order that they are listed, which is great if you would like more fine grained control. This can be very helpful if one file is retrieving data or providing setup that is used by the second file.
So now our file structure looks like this:
/myApp |- main.py | |- myFile.py | |- /Dir1 |- __init__.py | |- file1.py | |- /Dir2 |- __init__.py | |- file2.py
So what we have done here is add a file named
__init__.py into each directory that has no contents, it is an empty file. Although we can (and will in a little bit) add some contents to our
__init__.py, this empty file is perfectly valid to Python.
So with an empty
__init__.py file here is how we can use it.
# file1.py def firstFn(): print('Hello from file1.py') def secondFn(): print('Hello again from file1.py')
# file2.py def fnFile2(): print('Hello from file2.py') def secondFnFile2(): print('Hello again from file2.py')
# main.py import Dir1.file1 Dir1.file1.firstFn() # prints 'Hello from file1.py' Dir1.file1.secondFn() # prints 'Hello again from file1.py'
Or we can do this ...
# main.py from Dir1 import file1 from Dir1.Dir2 import file2 file1.firstFn() # prints 'Hello from file1.py' file1.secondFn() # prints 'Hello again from file1.py' file2.fnFile2() # prints 'Hello from file2.py' file2.secondFnFile2() # prints 'Hello again from file2.py'
The first thing to point out is we are using a dot notation
. to seperate our elements. Which allows us to 'chain' the directory to file to function. Although this is verbose and some people may not like the length of having to write everything out, it does provide clarity to the statement and does not try to hide anything.
5. Code in one
Inside our Dir1
# __init__.py from .file1 import firstFn, secondFn from .Dir2 import file2
Note that we have the above code only in our
Dir1/__init__.py file and our
Dir2/__init__.py file is still empty.
# main.py from Dir1 import firstFn, secondFn, file2 firstFn() # prints 'Hello from file1.py' secondFn() # prints 'Hello again from file1.py' file2.fnFile2() # prints 'Hello from file2.py' file2.secondFnFile2() # prints 'Hello again from file2.py'
6. Code in both
For our last example we alter the second line in the
Dir1/__init__.py file and add some code to
Inside our Dir1
# __init__.py from .file1 import firstFn, secondFn from .Dir2 import fnFile2, secondFnFile2
Inside our Dir2
# __init__.py from .file2 import fnFile2, secondFnFile2
# main.py from Dir1 import firstFn, secondFn, fnFile2, secondFnFile2 firstFn() # prints 'Hello from file1.py' secondFn() # prints 'Hello again from file1.py' fnFile2() # prints 'Hello from file2.py' secondFnFile2() # prints 'Hello again from file2.py'
So with this last example you can see that our result is being able to call each function from its nested file without having to use 'method chaining' and it gives us a much cleaner look. Although I would argue that it introduces some magic in the sense that it hides some of the complexity. But this is the balance that you need to decide on being a software engineer.
Thank you for reading this far and HAPPY CODING!!!