---

For this article, I assume that you the reader already know about working with directories and files. So that’s not what this article is about. Instead, I will be laying out a few general guidelines that will help ensure that your Python script will run successfully on any operating system.

Knowing where to save files.

One of the hardest problems about writing a Python script that will run on any operating system is knowing where you can, and can’t, save files. Different operating systems save files in different directories. So it becomes a challenge to determine the correct file path for different systems. There are 3 directories that are important; ROOT, HOME, and Documents. Let's take a few minutes to discuss each.

The ROOT directory is the directory where the running script is saved. This directory may, or may not, be a sub-directory of the user’s HOME directory.

The user’s HOME directory is the location where the user can save their files. Any script running under the user’s ID, will have access to read/write to files in this directory, and any sub-directories under it.

Many operating systems will create a Documents sub-directory under the user’s HOME directory. This is where things like Word documents, spreadsheets, and general data would be saved. Having such files in one directory makes it easier to find files and do backups.

But not all operating systems create a Documents directory. In such a case, the user’s HOME directory is used to save these kinds of files.

The following code snippet can be used to determine the path for these 3 directories, on any operating system.

import os

# Returns the path to the applications ROOT directory.
# This is the directory were the script has been saved.
ROOT_DIR = os.path.dirname(os.path.abspath( __file__ ))

# Returns the path to the users HOME directory.
HOME_DIR = os.path.expanduser('~')

# Returns the path to the Documents folder.
DOCUMENTS_DIR = os.path.join(HOME_DIR, "Documents")
if not os.path.isdir( DOCUMENTS_DIR ): DOCUMENTS_DIR = HOME_DIR

---

Always use the os.path.join() function, instead of string concatenation, for files & paths.

In some operating systems, the “Directory Separator Character” is the backslash ‘\’, and in others it’s the forward slash ‘/’.

So if you are accustomed to working in Windows, you might be tempted to create your file path using simple string concatenation, like this:

filename = os.path.expanduser('~') + “\” + “MyFile.txt”
with open(filename, ‘a’) as output:
   output.write(“This is a test.\n”)

But this will crash on a Linux system, because the “Directory Separator Character” is wrong.

So instead of using string concatenation, you should always use the os.path.join() function.

This will create a file path that will work on whatever the current operating system is being used.

filename = os.path.join( os.path.expanduser('~'),  “MyFile.txt” )
with open(filename, ‘a’) as output:
   output.write(“This is a test.\n”)

---

Whenever possible, use the full path in the filename.

When a Python script starts, the ‘current directory’ is whatever directory the shell launched the script from. Usually, this will be the directory where the script has been saved (the ROOT directory). But this is NOT always the case. So you should never assume that the current directory is the right directory.

Consider the following code examples. The first example assumes that the configuration file is in the current directory, assuming it to be the ROOT directory. But what if it’s not? Then your script won’t find the right file.

It is always better if you do as shown in example 2, to specify the full path in the filename, so that there is no confusion as to where the file might be found.

That’s not to say that you should never specify files as being in the current directory. Consider if you have multiple directories, each one containing the companies invoices for one day (multiple invoice files). Then your Python script might need to read all the invoice files in the current directory (the invoices for that particular day). So in this case, not specifying the full path in the filename might be appropriate.

So my guideline is this: Whenever possible, use the full path in every filename.

import os
import json


# Example1
# This is the WRONG way to do it.
# Here we are assuming that the config file is in the current dir.
# This will cause problems at some point in the future.

print( "Current Direcotry: ", os.getcwd() )

filename = "config.json"

if os.path.isfile(filename):
  with open(filename, "r") as InputFile:
    config = json.load(InputFile)
else:
  print(“The configuration file does not exists. The default settings are being used for now.”)
  config = {}



# Example2
# Here we are being very specific as to where to find the config file.
# This is a far better way to do it.

ROOT_DIR = os.path.dirname(os.path.abspath( __file__ ))
print("ROOT Directory: ", ROOT_DIR)

filename = os.path.join(ROOT_DIR, "config.json" )

if os.path.isfile(filename):
  with open(filename, "r") as InputFile:
    config = json.load(InputFile)
else:
  print(“The configuration file does not exists. The default settings are being used for now.”)
  config = {}

---

Always assume that filenames are NOT case sensitive.

Consider for a few minutes the following code snippet. Most operating systems are case sensitive, meaning that upper and lower case letters are completely different characters, and therefore these three filenames point to different files.

But there are a few operating systems that are NOT case sensitive, meaning that upper and lower case letters are the same character, and these three filenames all point to the exact same file. So if your script overwrites the file pointed to by filename02, then the other two files are also overwritten.

As a general guideline, your script should have a unique name for each file, and avoid mixing upper/lower case letters in filenames.

HOME_DIR = os.path.expanduser('~')
Filename01 = os.path.join( HOME_DIR, “myfile.txt”)
Filename02 = os.path.join( HOME_DIR, “MyFile.txt”)
Filename03 = os.path.join( HOME_DIR, “MYFILE.txt”)

---

Avoid using Unicode characters in filenames.

Almost all operating systems these days support filenames containing Unicode characters. These are filenames that contain things like Latin, Japanese, or Chinese characters, Math symbols, Emoticons, and so forth. A detailed discussion on this subject can be found at:

https://docs.python.org/3/howto/unicode.html

However, there are still a few operating systems that do NOT support Unicode filenames. And not all that do support Unicode characters, do so in the same way. These are usually older Unix or DOS based systems. They are rare, but they still pop-up from time to time.

So to ensure that your Python script will run on any operating system, my guideline is that you avoid using any Unicode characters filenames.

---

Never use spaces in your filename.

Even though most operating systems allow it, you should never use spaces in your filenames. The reason why is if you pass a filename to another script (see section about temp files bellow) the space character is used as a separator between parameters.

Even if you never plan to pass files between scripts, it’s best to NOT allow yourself to slip into the habit of using spaces in filenames. This is just one of the things that falls under “best practices guidelines”.

The following filename will technically work, but can cause problems latter on:

BadFileName = “This is my main data file.dat”

This is a far better way to name your files:”

GoodFileName = “MainData.dat”

---

A well written Python script will always check to see if a file or sub-directory exists before it tries to use it. Never assume that the file or sub-directory has already been created.

Let’s say that your script needs to read a file in a sub-directory where files for Project32 are saved. Everything works fine if the sub-directory has already been created.

But then you try to run the script on a device that does not already have the sub-directory, your script crashes with a “folder not found” error. Leaving the user with no clue as to what the problem is or how to fix it.

A well written Python script will anticipate problems like this, and will have the necessary code to correct them automatically.

If your project consists of multiple scripts, each script should check before file access is attempted. Never assume that the scripts have been run in the correct order, or that the files have not been somehow deleted between scripts.

Consider the following code snippet:

import json
import os

HOME_DIR = os.path.expanduser('~')
PROJECT_DIR = os.path.join(HOME_DIR, “Project32”)
ConfigFieName = os.path.join(PROJECT_DIR, config.json)

if not os.path.isdir(PROJECT_DIR):
  print(“The folder Project32 does not exists. It is now being created.”) 
  os.mkdir(PROJECT_DIR)

if os.path.exists(ConfigFieName):
  config = json.load( ConfigFieName)
else:
  print(“The configuration file does not exists. The default settings are being used for now.”)
  config = {}

---

Working with temp files.

There are times when you will need to use a temp file to work with a large amount of data, or to pass data between two scripts. I usually try to avoid using them, but I acknowledge that they can be useful under the right circumstances.

Temp files are usually deleted automatically when the script that created them terminates, or when the system reboots.

Each operating system handles temp files differently, making it very difficult to write low-level code for all operating systems to work with them.

Luckily, Python has a library for dealing with temp files, that takes all the hard work out of the problem. The name of that library is easy to remember: ‘tempfile’.

This is one of the ‘standard libraries’ (sometimes called ‘standard modules’), meaning that it comes pre-packaged when you install Python on your system. Nothing additional needs to be downloaded or set-up in order to use it.

And Yes, it even comes per-packaged with Pydroid for Android cell phones.

The official Python documentation for this library can be found at:

https://docs.python.org/3/library/tempfile.html

This documentation also includes examples.

The two Python scripts bellow, show how to use a temp file to pass data between two scripts. Please feel free to uses this code in your own projects. Note that the temp file created by script1.py will automatically be deleted when that scripts terminates.

In this example, I am passing a short string of text. But it could just as easily be a large document, a list of commands to execute, or anything else. The point is, I could uses a temp file to pass large amounts of anything between scripts.

# Script1.py

from subprocess import Popen, PIPE
import tempfile
import os
import sys

# Define the file holding the second script.
Script2Filename = os.path.join(os.path.expanduser('~'), "script2.py")

# Make sure that the second script exists.
if not os.path.exists(Script2Filename):
  print("Unable to find file: ", Script2Filename)
  print("This file is required to continue.")
  print("Script1.py ending with errors.")
  sys.exit()

# Pass some data to the second script using a temp file,
#  and print the reply (stdout) from the second script.
with tempfile.NamedTemporaryFile() as MyTempFile:
  print( "The name of the temp file is: ", MyTempFile.name )
  MyTempFile.write(b"This is the data being passed to the other script.\n")
  MyTempFile.flush() # flush the buffer.
  
  # Kick off the second script.
  process = Popen(["python3", Script2Filename, MyTempFile.name], stdout=PIPE, stderr=PIPE)

  # Wait for the second script to complete.
  stdout, stderr = process.communicate()

  # Print the results.
  print("stdout: ", stdout)
  print("stderr: ", stderr)

# script2.py
# This is the example script that is called by script1.py.

# Note: This file is saved in the users HOME directory.

import sys
import os

# Make sure some argument was passed.
if len(sys.argv)>0:
   MyScriptName = str(sys.argv[0])
   MyTempFile   = str(sys.argv[1])

   # Make sure that the temp file exists.
   if os.path.exists(MyTempFile):

      # Open the file, read the data. 
      with open(MyTempFile,"r") as InputFile:
        PassedData = InputFile.read()
        print("The passed data: ", PassedData)
        

So my guideline for temp files is this:

If you need to use a temp file, DON’T try to write code to do it yourself. Use the tempfile library instead.

---

Writing a block of code for different operating systems.

There are times when your script will need to determine what operating system it’s running on, and execute a different block of code for each one. The sys.platform method will return a string of lower case text, telling what operating system is being used.

Information about the sys.platform method can be found at the official Python documentation page:

https://docs.python.org/3/library/sys.html#sys.platform

As of the time that I an writing this, I know of 14 possible values that this method might return. More may be added as new operating systems become available.

They are:

AIX:			aix
FreeBSD 7:		freebsd7            
FreeBSD 8:		freebsd8            
FreeBSD N:		freebsdN            
Linux:			linux
Linux: 			linux2 
OpenBSD 6:		openbsd6        
OS/2:			os2                 
OS/2 EMX:		os2emx              
RiscOS:		riscos              
AtheOS:		atheos              
Windows:		win32               
Windows/Cygwin:	cygwin              
Windows/MSYS2:	msys                

NOTE: The sys.platform method will return the string ‘linux’ or ‘linux2’, for Android and Raspbian operating systems. So the values returned are of a very generic nature.

import sys

OS_TYPE = sys.platform

if OS_TYPE.startswith('freebsd'):
    print( "This system is running a version of FreeBSD.")
    # Additional logic for FreeBSD goes here.
elif OS_TYPE.startswith('linux'):
    print( "This system is running Linux.")
    # Additional logic for Linux goes here.
elif OS_TYPE == 'aix':
    print( "This system is running AIX.")
    # Additional logic for AIX goes here.
elif ("."+OS_TYPE+".") in ".win32.cygwin.msys.":
    print( "This system is running Windows.")
    # Additional logic for Windows goes here.
elif OS_TYPE == 'darwin':	
    print( "This system is running Mac/OS.")
    # Additional logic for Mac/OS goes here.
else:
    print( "This system is running a different type of OS.")
    # Additional logic for OTHERS goes here.

---

In concussion.

And that concludes what I have to say about this subject. I hope that someone out there find’s this useful.

Everyone have a good day, and be kind to each other.

Joe Roten. www.gsw7.net/joe

Last updated: 2021-02-12