---

Quickstart.

If you are not interested in reading all the documentation, and are only looking for code that you can quickly cut & past into your project;

Step 1. Download the 4 files (links found at the bottom of this doc).
Step 2. Start the server (ShareServer.py) and let it run in the background.
Step 3. Use client7.py, client8.py, and share.py to build your clients.
Step 4. Come back to this document if you have any questions.

What is it?

First, let me say that this document is not intended to be a ‘you should do it this way’ kind of instructions.

Rather, this is to document how I did it, and to invite you the reader to consider this in your own projects. Please feel free to modify and uses my code to your liking. You could use it as-is, or as a starting point for something completely different.

Some may find my documentation rather lacking. Sorry. I’m the kind of programmer who feels that a few lines of sample code can be worth a few pages of written documentation.

ShareServer is a python script that I have written, that can be used to share data, and pass messages between other programs ( called the clients ). It can also be used as a simple database server, or to process complicated math problems that he client’s can’t handle.

The clients don’t have to be written in python, or be running on the same computer as the server. The clients can be on Linux, Windows, Mac. The clients can even be running on an Arduino, ESP32, Raspberry Pi, or other Single Board Computer (SBC). Just as long as the client can create a network ( sockets ) connection to the server.

ShareServer can also be a starting point for you to create your own customized server. I tried to make this as easy to understand and modify as possible, so you can create customized servers to fit your own needs.

Starting the server ( for people new to python )

So to begin my tour of what I have created:

You will need to have python installed on your computer. If you don’t, it can be installed from www.python.org.

Next, you will need to download ShareServer.py , click here to download.

To start ShareServer, open a terminal window ( or a command prompt if you are using Windows ), and enter this command:
python ShareServer.py

On some versions of Linux, you might need to uses this command instead:
python3 ShareServer.py

If all goes well, you will see: Server is now listening on port 2980.

Don’t close the window! Just minimize it so that the server runs in the background.

How does this server work?

The client sends a python expression to the server as a string of text. The server then evaluate the expression, and the result is returned to the client as another string of text.

Some may find that last statement is about as clear as a large block of concrete. So I am including the following example code. This code will preform operations like sending a text message to all the clients, and retrieving a message from our client’s inbox. It will also show you on the screen the text that is being sent to the server, and the reply coming back from the server.

Several commands in this example, like exit() and getUserID(), are actually python functions defined inside ShareServer. We can include them inside the python expression being sent to the server, and whatever value these functions return, is the string that is returned from the server.

"""
client8.py
written by Joe Roten

This is a demonstration of using the server ShareServer.py.
It will show the string of text sent to the server, 
and the reply from the server.

In order for this to work, ShareServer.py must be
running on the same computer as this client.

For documentation, see
   https:\\www.gsw7.net\K700009.php

"""

import socket

def SendToServer(host, port, request):
    """Send a request to the server, and return the reply."""
    reply = ""
    try:
      client_socket = socket.socket()
      client_socket.settimeout(20)
      client_socket.connect((host, port))
      client_socket.send( request.encode() )
      reply = client_socket.recv(1024).decode()
      client_socket.close()
    except:
      reply = "Error"
    print("   Sent to server: ", request)
    print("Reply from server: ", reply )
    print("")
    return reply


host = "localhost"
port = 2980
cmdGetID = "getClientID()"
cmdSend = "send('{}', '{}', '{}')"
cmdRecv = "recv('{}')"
cmdExit = "exit('{}')"


# Check that the server is responding.
SendToServer( host, port, "'Hello World'" )
SendToServer( host, port, " ( 1 + 1 ) * 2 " )
SendToServer( host, port, " math.log10( 134256 ) " )

# Get a client ID and regester with the messaging system.
ID = SendToServer( host, port, cmdGetID )

# Send a test message to all clients on the server.
# Note: 'ALL" is a predefined group; all clients are a member.
cmd = cmdSend.format( ID, "ALL", "This is a test." )
SendToServer( host, port, cmd )

# Retrive the next pending message in my client's inbox.
reply = SendToServer( host, port, cmdRecv.format( ID ) )

print("The message is: ", reply.split( chr(200), 2 ) )
print("")

# Release any server resources used by this client.
SendToServer( host, port, cmdExit.format( ID ) )

# end of code.

Limits to the number of clients.

In theory, the max number of clients is only limited by your network bandwidth, your CPU speed, and available memory on the host computer.

True, ShareServer can only service the request of one client at a time. But each request should take only a fraction of a second, and the clients will wait ( timeout ) for 20 seconds before giving up the connection.

I have personally had over 60 clients, with a Raspberry Pi 3B as the server host.

And that’s over a very slow WiFi connection.

About share.py

Share.py is a python module that I have written to make it easier to write clients in python. It contains the ShareServer class, which does two things:

1. It can search the local network for the running ShareServer.py so we don’t have to worry about finding the server’s IP addresses.

2. It’s a wrapper for many of the commands that can be used with ShareServer.

It is very simple, and is intended to be easily ported to other programming languages, like C or Basic.

Do you reallyneed share.py to create a client? No, I didn’t use it for the above example.

Does it make writing clients easier? Yes.

It can be download from my website by clicking herea>.

Then, take a fewminutes to look this code over, and the upcoming example which uses it.

There is not much need in documentation on this class, the code is pretty self explanatory.

Messaging System & Messaging Groups.

The Messaging System is a collection of python functions and SQLite tables that allow clients to pass messages between each other. Each client is identified to the Messaging System by a unique client ID.

Messages being sent to a client are stored in the client’s messaging queue ( or inbox ) until the client requests the next pending message.

The Messaging System is client driven; meaning that the next pending message from the inbox, is sent to the client only when the client requests it.

Messages are sent to another client using the send() function, and the next pending message is retrieved using the recv() function. These functions are defined in the ShareServer class in share.py.

The recv() function returns the next pending message in the form of a python truple in the format:

( sentFrom, sentTo, Text ). Where sentFrom is the group name or client ID of the sender, sentTo is the group name or client ID the message is being sent to, and text is the contents of the message as a string.

If there are no pending messages in the client’s queue, the recv() function will return the values
(‘None’, ‘None’, ‘None’,)

The client7.py (bellow) shows several examples of using these functions.

NOTE: These functions can also be called directly from ShareServer.py, bypassing the wrapper in share.py, as shown in client8.py (above) . But if you chose to do that, please note that the syntax of these functions is different than the wrapper. See ShareServer.py for the syntax of these functions if used as a direct call. I apologize if this causes confusion.

A Messaging Group is a collection of one or more client IDs. We can use it to send a text message to 1 or more clients. Example:

sh.send( “group16”, “This is a message sent to everyone in group16.” )

A client can join a messaging group by using the join() command. Example:
sh.join( “group16” )

The name of a messaging group
is case sensitive,
should be 60 characters long or less,
and can include spaces.

The name of a messaging group should NOT include periods “.”, or dashes “-”.

A new group is created automatically when someone joins it.

There are no limit to the number of groups, or to the number of clients within a group.

There are a few predefined messaging groups:
‘ALL’ - All clients are automatically a member of this group.
‘NOPLY’ - Means that the clients can’t send a reply to messages coming from this group.
‘SYSTEM’ - Used to send commands to the server.
‘CONSOLE’ - Used to send messages to the Operators Console ( if one is used ).
‘SYSADMIN’ - Used to send message to the system admin ( if there is one ).
‘SQLite’ - Used to send and receive messages to/from the database engine.

Each client has it’s own messaging queue ( or inbox ). This is so that messages intended for ‘John Smith’ are separated from messages going to ‘Jill Jones’. But if both John and Jill are members of group16, they will both receive a copy of any messages sent to that group.

Best Practices:
A group can contain just 1 client, or many. So you could have a group named ‘John Smith’, with only 1 client in that group. That way, sending a text message to group ‘John Smith’ is much easier to remember than client ID ‘K467Bu92’ is John’s. Also, John can send a message from group ‘John Smith’, and that name shows on the receivers console, rather than the confusing ‘K467Bu92’.

IMPORTANT: After 10 minutes of inactivity between a client and the Messaging System, the client is assumed to have terminated, and any pending messages in the client’s queue WILL BE LOST. It is therefore very important that a client check for new messages at least once every 7 minutes, using the recv() function.

Sharing data between clients.

One of the more useful things that ShareServer can do for you, is for your clients to share variables. Imagine that client A crates an int variable called TheAnswer, and client B (not running on the same computer) can read and even change the value. It is as if all your clients can create variables in the cloud, or in some kind of shared memory space.

The WRONG way to do it. Let me start by showing you the wrong way to do this. The wrong way would be to simply define a new variable, which would exist in the global scope of the server. All clients would then be able to access or change the value of these variables. Technically it will work, but it opens the door to a number of problems. Consider what would happened if you accidentally used a variable name that is already being used by ShareServer.

# Code on client A
sh.SendToServer(“ TheAnswer = 42  “)
sh.SendToServer(“ Address1 = ‘1313 Mocking Bird Lane.’ ”)

# Code on client B
sh.SendToServer(“ Address1 = ‘206 Hemingway Ave.’ “)
print(  sh.SendToServer(“ TheAnswer “ ) )

A better way to do it. ShareServer has a python dictionary object called MyData, that is intended for passing data between clients. Not only does this isolate our data from the global scope of the server, it also allows us to use periods “.” in the key names (a style known as dot notation).

Adopting this style of dot notation for key names, allows us to add structure to our data schema, and also allows you to use the very powerful fetch() and delete() functions, which will be covered a bit later.

Unfamiliar with the python dictionary data structure? Click the link to learn more.

# Code on client A
sh.SendToServer(“ MyData[‘TheAnswer’] = 42  “)
sh.SendToServer(“ MyData[‘project17.Address’] = ‘1313 Mocking
Bird Lane.’ ”)
sh.SendToServer(“ MyData[‘project17.encription.key’] =
‘Bgj7G42554’  “)
sh.SendToServer(“ MyData[‘garden.soil.temp’] = 56  “)

# Code on client B
print( sh.SendToServer(“ MyData.get(‘TheAnswer’, ‘I don’t
know.’) “ )
print( sh.SendToServer(“ MyData.get(‘garden.soil.temp’, ‘I
don’t know.’) “ )
sh.SendToServer(“ MyData[‘project17.Address’] = ‘206
Hemingway Ave.’ ”)

An even better way. There are two functions that we can uses to Add, Retrieve, and Change the values in MyData, but make for a much cleaner and more structured code. These functions are setReg() and getReg(). The example bellow does the exact same thing as the example above, but consider how much easier it is to read and troubleshoot any issues.

# Code on client A
sh.setReg( ‘TheAnswer’, 42 )
sh.setReg( ‘project17.Address’,  ‘1313 Mocking Bird Lane.’  )
sh.setReg( ‘project17.encription.key’], ‘Bgj7G42554’  )
sh.setReg( ‘garden.soil.temp’,  56 )

# Code on client B
print( sh.getReg( ‘TheAnswer’, ‘I don’t know.’)
print( sh.setReg( ‘garden.soil.temp’, ‘I don’t know.’)
sh.setReg( ‘project17.Address’,  ‘206 Hemingway Ave.’)

Using fetch() and delete()

.

There are two more functions that can be used for working with the server’s MyData dict, they are fetch() and delete().

The delete() function will remove all entries in MyData who’s key matches a given pattern.

The fetch() function will send to the client, using the Messaging System, all entries in MyData who’s key matches a given pattern.

Both of these functions use the fnmatch() function to determine if a key is a match or not.

This is why adopting the uses of dot notation in your key names is such a powerful concept. It allows you to work with groups of entries, as if they were a single thing.

Unfamiliar with how the fnmatch() function works? Click the link to learn more.

# Delete all information from MyData for project17.
sh.delete( "project17.* ")

# Show all the server’s info, in MyData, on the screen.
sh.fetch( “server.*” )
sentFrom = ""
while sentFrom != “None”:
    (sentFrom, sentTo, text) = sh.recv()
    print( text.split(“:”))

Using ShareServer as a database server

ShareServer can be used as a database server, much the way that a MySQL server works, but without the need to install any driver software on the client system.

ShareServer uses SQLite3 as a database engine. SQLite3 is installed automatically when you install python from www.python.org, so it is probably already installed on your host computer.

SQL stands for Structured Query Language , which is a programming language used to work with large data sets. And Yes, ShareServer supports it (see example code bellow).

The SQL language is pretty easy to learn. Doing a google search for ‘SQLite’ will show you many websites where FREE documentation and examples are available. Most python programmers can learn SQL in less than an hour.

You can use the dbExec() function, which is part of the ShareServer class (found in share.py), to easily execute SQL commands on the server. Example:

sh.dbExec( “INSERT INTO mydata (name, address) VALUES (‘Jill Jones’, ‘123 Main Street’)”)

You can also also execute SQL commands by sending a text message to group ‘SQLite’.

sh.send( ‘SQLite’, “SELECT * FROM mydata WHERE name=’Jill Jones’; “)

If the SQL command you send to the server starts with the word “select”, it assumes that some information needs to be sent back to the client. It does this by sending text messages using the Messaging System. Each record (row) of the data returned, is sent as a separate message. These messages will come from group “SQLite”. The text of the message can be changed into a python truple, using the python built in eval() function. Example:

sh.send( “SQLite”, “SELECT id, name, address FROM mydata WHERE state=’tx’;”)
sentFrom = “”
while sentFrom !=”None”:
  (sentFrom, sentTo, text) = sh.recv()
   try:
     print(“”)
     print(“text: “, text)
     record = eval(text)
     print(“id: “, record[0] )
     print(“name: “, record[1] )
     print(“address: “, record[2] )
  except:
      pass

Putting it all together.

OK, now that that documentation is out of the way, it’s time to show you what this can really do!

Consider for a few minutes the code example bellow. It’s kind of long, but gives some ideal of what can be done.

"""
client7.py
written by Joe Roten

This script will deminstrate a few of the functions of ShareServer.py

This requires that ShareServer.py to be running either on the same computer,
or on a computer on the same subnet ( on the same WiFi router ).

The file share.py should be in the same folder as this script.

For documentation, see
   https://www.gsw7.net/K700009.php

"""

import share
import sys

def ShowMyQueue():
    """Show the messages in my queue on the screen."""
    sentFrom = ""
    while sentFrom != "None":
      (sentFrom, sentTo, text) = sh.recv()
      if sentFrom != "None":
        print("sentFrom: ", sentFrom)
        print("sentTo:   ", sentTo)
        print("text:     ", text)
        print("")

def ShowFetchReturns():
    """Show the messages in my queue created by fetch() or dbExec()."""
    sentFrom = ""
    while sentFrom != "None":
       (sentFrom, sentTo, text) = sh.recv()
       if sentFrom != "None":
           print(text)
    print("")


#####################################################
#
# Housekeeping.
#
#####################################################

sh = share.ShareServer()

if not sh.isServerUp():
  print("Sorry, the server is not responding.")
  sys.exit()
else:
  print("The server is responding...All systems are GO!\n")

print( "My client ID: ", sh.ID, "\n" )

#####################################################
#
# Tell the system who I am (optional)
#
#####################################################
MyName = "John Smith"
sh.join( MyName )

#####################################################
#
# Join a few more messaging groups (optional)
# See the documentation for more info about groups.
#
#####################################################
sh.join( "group16" )
sh.join( "Chcago Office" )
sh.join( "managers" )
sh.join( "sysadmin" )

#####################################################
#
# Examples of sending messages to other clients or groups.
# Each client has their own message queue.
# send() will copy a message to one or more of these queues.
# The syntax is:  send( sentFrom, sentTo, text )
#
# Note: Messages to Jill & console3 will not be copied to my queue.
#   I can still send to these groups OK, but I am not Jill or
#   a console operator, and don't want to be copied on
#   their messages. Since I didn't join these groups, I won't
#   recieve them.
#
# Note: 'ALL' is a pre-defined messaging group.
#   All clients are automatically joined to the 'ALL' group.
#
#####################################################
sh.send( sh.ID, "ALL", "This is a test sent to everyone." )
sh.send( sh.ID, "group16", "This is a test sent to everyone in group16." )
sh.send( sh.ID, "sysadmin", "This is a test sent to everyone in sysadmin." )
sh.send( sh.ID, "console3", "This is a test sent to everyone in console3." )
sh.send( sh.ID, sh.ID, "This is a test sent to ME." )

sh.send( MyName, "ALL", "This is a test sent to everyone." )
sh.send( MyName, MyName, "This is another test sent to myself." )
sh.send( MyName, "Jill Jones", "This is test sent to Jill." )

# Sending a ++ping is a good way to list clients in a group.
sh.send( sh.ID, "group16", "++ping, This is a test ping." )

# Now, Show all the messages that are waiting in my queue.
ShowMyQueue()

#####################################################
#
# Examples of setReg() and getReg().
# All clients can access, create, change, and delete,
#   all registry entries ( share data ).
#
#####################################################
sh.setReg( "project17.value1", 3476)
sh.setReg( "project17.value2", 1236)
sh.setReg( "project17.address", "1313 Mockingbird Lane, Chicago IL, 99999")
sh.setReg( "project17.isRunning", True)

# To change a value, just uses setReg() again.
sh.setReg( "project17.value1", 123574)

# Examples of retrieving values.
print( "project17.value1: ", sh.getReg("project17.value1") )

if bool( sh.getReg("project17.isRunning")):
   print( "project17 is running.")
   
print("")

#####################################################
#
# Using fetch() to get all reg values for project17.
# Results are sent to my queue as text messages.
# Note the use of the [*] wildcard.
#
#####################################################
sh.fetch( "project17.*" )
ShowFetchReturns()

#####################################################
#
# Using fetch() to get all the server's information.
#
#####################################################
sh.fetch( "server.*" )
ShowFetchReturns()

#####################################################
#
# Using delete() to delete all reg values for project17.
#
#####################################################
sh.delete( "project17.*" )

#####################################################
#
# Using dbExec() to preform SQLite commands on the server.
# Yes, this can be used as a database server.
#
#####################################################

sh.dbExec("SELECT datetime();")
ShowFetchReturns()

sh.dbExec( "CREATE TABLE IF NOT EXISTS testing (id integer PRIMARY KEY, field1 TEXT);")

for x in range(0,10):
  text = "This is record " + str(x)
  sh.dbExec("INSERT INTO testing (field1) VALUES ('{}');".format( text ) )
  
sh.dbExec("SELECT * FROM testing;")

# The results of the above SELECT is sent as text messages.
# Each row is one message.
ShowFetchReturns()

sh.dbExec("DROP TABLE IF EXISTS testing;") 

#####################################################
#
# Cleaning up the client's data and shutting down.
# This will free resources on the server and reduce
#   lag for the other clients.
#
#####################################################
sh.exit()

sys.exit()

# End of client7.py

Customizing ShareServer.

You can add functionality to ShareServer by adding client callable functions. You could for example, add a function to record a message to a log file on the server, or to download a file from the internet, or retrieve the weather forecast from the National Weather Service, etc...

First, make a backup copy of ShareServer.py. That way, if something goes wrong, you have a good working copy to fall back to.

Now, take a look at the code in ShareServer.py and find where it says “Client callable functions.”. These are the functions that the client can call.

At the end of this chunk, you will find “Additional functions can go here”. Add your functions, using the ones I have created as examples. It really is just that easy.

It is usually a good ideal to also add to the doc string (the area at the top of the file that says who wrote the script and when) to document who made any change, when and why. You should always leave a documentation trail.

Be sure to save the file before closing.

Finally, modify share.py to add wrappers for your new functions. This will make writing new clients much easier.

Don’t forget to restart ShareServer.py so that your changes go into effect.

Complex operations on an Arduino Uno ??

Lets say that you have a project requiring you to do complex math on an Arduino. The Arduino can’t do trig or log functions, but it CAN send a string of text to another device. Do you see where I’m going with this?

The Arduino simply passing a string of text to the server, and the server does all the heavy work. You also have access to ShareServer’s database and messaging functions as well.

Examples:

Answer = SendToServer( host, port, “ math.log10( 127645 ) * 2.5 “ )
SendToServer( host, port, “ setReg( ‘garden.soil.temp’ , 56 )” )
dt = SendToServer( host, port, “datetime.datetime.now()" )

Consider using a Raspberry Pi as your server.

Here’s an ideal. Instead of using your desktop system as the host, why not use a cheep Raspberry Pi?

It could be setup next to your WiFi router, running in a headless configuration.

Out of the way, out of sight, low power, and always available to service the client’s requests.

Download links.

Here again are the download links for the files I’m covering today. Click on the filenames to download.

ShareServer.py
share.py
client7.py
client8.py

In concussion.

I have spent way more time writing this document than I had intended. So its time I move on to my next task.

Sorry that the documentation is so slim, but I’m sure you can figure it out.

Everyone have a good day, and be kind to each other.

Joe Roten. www.gsw7.net/joe

Last updated: 2020-09-15