Bluetooth Communication with Palm OS -
Processing and displaying the collected data (Part 2/2)

06.06.2020

In the last part, we configured a PHP-Server, which is now able to deliver the data, which needs to be displayed on a (bluetooth enabled) Palm PDA. So a connection is needed to transfer the collected data to the Palm. An ESP32 is perfect for transferring the data from the PHP server to the Palm, since it is able to talk "legacy" bluetooth and can access the Wifi simultaneously.

This article is grouped in two parts: this first part is about the setup of the "transfer-unit" and second part about wirting the Palm OS Application, which displays the fetched data on a PalmOS device.

Needed parts

These parts are required for transferring the collected data and display it on a Palm:

A bluetooth enabled Palm OS device (e.g. a Tungsten T3, but also Palm OS 4.1 devices with a Bluetooth SDIO card will work.)
One ESP32 (here an "ESP32 Dev Module" is used)

Setting up an ESP32

At this point, I will again skip the part of explaining how to set up an ESP32 with the Arduino IDE. There are already good tutorials, which explains this very well, like this one: https://randomnerdtutorials.com/installing-the-esp32-board-in-arduino-ide-windows-instructions/

Make sure these drivers for the ESP, like mentioned in the linked article, are installed: https://www.silabs.com/products/development-tools/software/usb-to-uart-bridge-vcp-drivers
Also these Arduino IDE Settings worked fine for me:

("Keine" can be translated with "none".)

Since there is no wiring needed for the ESP32 (except the power supply), the only important part is the code.

The code

The only task, which must be done by the ESP is to receive a request from the Palm, process it, requesting the needed data from the PHP server and send it back to the Palm.

#include <WiFi.h>
#include <BluetoothSerial.h>
#include <HTTPClient.h>


const char* ssid = "My Wifi SSID";
const char* password =  "secret-password";

BluetoothSerial SerialBT;
char *pin = "1234";

void setup() {

  Serial.begin(9600);
  
  delay(1000); // Power up

  SerialBT.begin("ESP32"); //Bluetooth device name
  SerialBT.setPin(pin);
  Serial.println("The bt-device started, now you can pair it with bluetooth!");

  Serial.println("Starting Wifi...");

  wl_status_t state;

  state = WiFi.status();
      
  while (state != WL_CONNECTED) {
      WiFi.mode(WIFI_STA);
      WiFi.begin(ssid, password);
      Serial.print(".");
      vTaskDelay (2500);
      state = WiFi.status();
  }      
  Serial.println("");
  Serial.print("Connected to ");
  Serial.println(ssid);
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());

}
void loop() {

  while(SerialBT.available()) {

    Serial.println("BT-Data available.");
    
    String sensor = SerialBT.readString();
    sensor.trim();
    Serial.println(sensor);
     
    if ((WiFi.status() == WL_CONNECTED)) {

      Serial.println("Wifi still connected.");
      
      HTTPClient http;
   
      http.begin("http://192.168.1.55/dataProvider.php?sensor="+sensor);
      int httpCode = http.GET();

      Serial.println(httpCode);
   
      if (httpCode > 0) { //Check for the returning code
         
          //http.writeToStream(&SerialBT);
          String payload = http.getString();
          payload.trim();
          Serial.println(payload);
          SerialBT.println(payload);
      }else{
        Serial.println("Invalid data.");
      }
      
      http.end(); //Free the resource
      
      
    }else{
      Serial.println("Need to restart Wifi...");
      WiFi.mode(WIFI_STA);
      WiFi.begin(ssid, password);
      
   }
  }
}

First we need to include the libraries for Wifi, Bluetooth and a http-client. In line 5 and 6, the credentials for the own Wifi are set. After instantiate the BluetoothSerial class, in line 9 is the pin defined, which is used for authentication. In line 14 the serial port is opened for the debugging-output of the serial monitor of the arduino IDE, we then give the script some time (line 16) to power up completely. With line 18 and 19, everything is prepared for a bluetooth connection. After this point only the wifi connection gets established.

In the loop-function, we wait for the request from the Palm PDA (line 44), after we got one, the requested data (from the corresponding sensor - line 48) is getting trimmed, so we get no unwanted whitespaces (line 49). After we checked, that the wifi-connection is still present (line 52) (or we restart the connection in the lines 79 and 80), we are using the HTTP client (line 56) to fetch the needed data from the PHP server (line 58 - 67). Here is important, that the correct IP address is provided (192.168.1.55 is only a sample IP). Then we simply redirect the data via the bluetooth connection to our Palm PDA (line 69).

The Palm OS Application

The last and the biggest part is about the Palm OS Application. We will establish a bluetooth connection to the ESP32, receiving data, processing it and display it on the screen. We will also handle lost connections. The complete project is available here: BluetoothComm source code and can be build with Codewarrior 9.3. (The debug-target has some problems, but the release target works fine.)

If you just want to try it out, here is the prebuild .prc file: bluetoothComm.prc
But keep in mind, that at least a ESP32, which sends some mocked data, is needed for a test.

The application is divided in four parts:

Main-Programm with all Palm OS specific functions (BluetoothComm.c)
Library with all bluetooth related functions (BtAppLib.c)
Library with all serial related functions (SrAppLib.c)
Library with all user interface related functions (UiAppLib.c)

BluetoothComm.c

/*
 * BluetoothComm.c
 *
 * main file for BluetoothComm
 *
 */
 
#include "BluetoothComm.h"
#include "BluetoothComm_Rsc.h"
#include "BtAppLib.h"
#include "UiAppLib.h"
#include "SrAppLib.h"
 
#include <PalmOS.h>
#include <PalmOSGlue.h>
#include <UI/UIPublic.h>
#include <System/SystemPublic.h>
#include <PalmCompatibility.h>
#include <PalmTypes.h>
#include <PalmOS.h>

// The actual start-point of the application,
// which also checks the OS version.
UInt32 PilotMain(UInt16 cmd, MemPtr cmdPBP, UInt16 launchFlags)
{
        UInt32 currentRomVersion;
        UInt32 requiredRomVersion;
        requiredRomVersion = 0x04000000;
        setErrorExit(0);

        FtrGet(sysFtrCreator, sysFtrNumROMVersion, ¤tRomVersion);
        if (currentRomVersion < requiredRomVersion)
        {
                FrmAlert(OsVersionTooLowAlert);
                return(sysErrRomIncompatible);
        }

        if (cmd == sysAppLaunchCmdNormalLaunch)
        {
                Err returnCode;
                returnCode = startApplication();

                if (returnCode != 0)
                {
                        ErrAlert(returnCode);
                        return(returnCode);
                }
                AppEventLoop();

                returnCode = stopApplication();
                
                if (returnCode != 0 && !getErrorExit())
                {
                        ErrAlert(returnCode);
                        return(returnCode);
                }
        }

        return 0;
}

// startApplication also starts the bluetooth stack as client
Err startApplication()
{
	Err returnCode = 0;

	returnCode = startBluetoothStack();
	if ( returnCode == 0) {
		FrmGotoForm(MainForm);
	}

	return returnCode;
}

// This is the main event loop 
static void AppEventLoop(void)
{
        EventType event;
        Err error;
       
        do
        {
                EvtGetEvent(&event, evtWaitForever);

				
                if (SysHandleEvent(&event)) {
                        continue;
                }


                if (MenuHandleEvent((void *)0, &event, &error)) {
                        continue;
                }


                if (AppHandleEvent(&event)) {
                        continue;
                }


                FrmDispatchEvent(&event);
                
        }
        while (event.eType != appStopEvent && !getErrorExit());
}

// This is the function with the business-logic
// This function handles all form events like (software) button presses
static Boolean MainFormHandleEvent(EventType *event)
{

    Boolean handled = false;
    Boolean btRequest = false;
    UInt8 waitCounter = 0;
    UInt8 maxCounter = 10;
    char* rawData = NULL;
    UInt16 tappedButton;
    
  	// The following strings must be identical to the ones in the config.php!
	char* bedroomPtr = "Bedroom"; 
	char* basementPtr =  "Basement";

    if(event->eType == frmOpenEvent) {

            FrmDrawForm(FrmGetActiveForm());
            ClearField(MainForm, DialogField);
			WriteInDialogField(MainForm, DialogField, "Please select a data point.");
            handled = true;
    }

    if(event->eType == ctlSelectEvent) {

            switch (event->data.ctlSelect.controlID)
            {

                case ClearButton:
                        ClearField(MainForm, DialogField);
                        handled = true;
                        break;
                        
                case BedroomBtn:
                		
                        SendDataSerial(bedroomPtr);
                        handled = true;
                        btRequest = true;
                        break;
                        
                case BasementBtn:
                        SendDataSerial(basementPtr);
                        handled = true;
                        btRequest = true;
                        break;
                        
                default:
                  		break;
      		}
    }
    
    if(btRequest){
    
    	ClearField(MainForm, DialogField);
    	WriteInDialogField(MainForm, DialogField, "Fetching data...");
    
	    while(!rawData){
	    
	        waitCounter++;
    		SysTaskDelay(sysTicksPerSecond/4);
	    
			// Read data from the serial port.
		    rawData = ReadSerial();
		    
			if(rawData && StrLen(rawData) > 20){
			
				ProcessSerialData(rawData);
				
			}
			
			if(getErrorExit()){
				break;
			}
			
			if(waitCounter >= maxCounter){
			    ClearField(MainForm, DialogField);
    			WriteInDialogField(MainForm, DialogField, "Fetching data... failed!");
    			
    			tappedButton = handleConnectionLost(BtErrorOccurredAlert3);
    			
    			if(tappedButton == 0 || tappedButton == 1){
    				// exit or reconnect, not more action here required
					break;
				}
    			
				if(tappedButton == 2){
					// retry
					ClearField(MainForm, DialogField);
    				WriteInDialogField(MainForm, DialogField, "Retrying to fetch data...");
					
					waitCounter = 0;
					maxCounter = 20;
				}
				

				
			}
		}
		

		
	}
    
    return handled;
}

// Handle all incoming events
static Boolean AppHandleEvent(EventPtr event)
{
        FormType* formTypePtr;
        UInt16 formId;
        Boolean handled = false;

        if (event->eType == frmLoadEvent)
        {

                formId = event->data.frmLoad.formID;
                formTypePtr = FrmInitForm(formId);
                FrmSetActiveForm(formTypePtr);

                switch (formId)
                {
                case MainForm:
                        FrmSetEventHandler(formTypePtr, MainFormHandleEvent);
                        break;


                default:
                        break;
                }

                return true;
        }

        if (event->eType == menuEvent)
        {
                MenuEraseStatus(NULL);
                switch (event->data.menu.itemID)
                {
                case MainOptionsHelpCmd:
                        FrmAlert(HelpAlert);
                        handled = true;
                        break;

                case MainOptionsAboutCmd:
                        FrmAlert(AboutApplicationAlert);
                        handled = true;
                        break;

                default:
                        break;
                }

                return true;
        }

        return false;
}

// stop the application and close all open ports
Err stopApplication()
{
        Err error;
        UInt16 btPort = getBtPort();
         
        ClearField(MainForm, DialogField);
		WriteInDialogField(MainForm, DialogField, "Shutting down bluetooth stack...");
       
		error = SrmSendWait(btPort);
		ErrNonFatalDisplayIf(error == serErrBadPort,"SrmClose: bad port");
		if (error == serErrTimeOut){
			FrmAlert(ErrorOccurredAlert);
		}
		SrmClose(btPort);
        FrmCloseAllForms();
        return error;
}

// Triggers an exit if an error occurs (if ErrorExit is not zero)
Err getErrorExit(){

	UInt32 error = 0;

	FtrGet(appFileCreator, 10110, &error);
	
	return error;
}
void setErrorExit(UInt32 error){

	FtrSet(appFileCreator, 10110, error);
}

// This function proccesses the serial data, which gets received by the ESP32
static void ProcessSerialData(char* rawData){

	UInt8 i;
	UInt8 c = 0;
	UInt8 j = 0;
	UInt8 k = 0;

	char complete[300] = "";
	char time[30] = "";
	char temp[15] = "";
	char hum[15] = "";
	
	// Not beautiful, but it is working.
	// I'm not sure if there is something like "explode(';',rawData);"
	// Since, every data has a fixed length, 
	// we can work with fixed values, too. 
	for(i = 0; i < StrLen(rawData); i++){

		if( ';' == rawData[i]){
			c++;
		}

		if(c == 0 && rawData[i] != ';' && i < 20){
			time[i] = rawData[i];
		}

		if(c == 1 && rawData[i] != ';' && j < 5){
			hum[j] += rawData[i];
			j++;
		}

		if(c == 2 && rawData[i] != ';' && k < 5){
			temp[k] = rawData[i];
			k++;
		}
	}
	  
	StrCat(complete, "Time: ");
	StrCat(complete, time);
	StrCat(complete, "\n");
	StrCat(complete, "Temperature: ");
	StrCat(complete, temp);
	StrCat(complete, "C\n");
	StrCat(complete, "Humidity: ");
	StrCat(complete, hum);
	StrCat(complete, "%\n");
	
	ClearField(MainForm, DialogField);
	WriteInDialogField(MainForm, DialogField, complete);	
}

The first function is the PilotMain. This is the start point of our application. Here we check if Palm OS 4 or higher is available. Since there is also a SDIO bluetooth card for Palm OS 4 devices with a SDIO slot, the application can also run on these PDAs (e.g. the Palm m125). After the check, startApplication() gets called, where the bluetooth stack gets started with: startBluetoothStack() This function is defined in the BtAppLib.c. The AppEventLoop simply receives new events and redirects them to the according functions. The most important is: MainFormHandleEvent since AppHandleEvent "only" handles the menu events. (Of course, this could be more important if we wanted it to be.) In the MainFormHandleEvent, we are telling the user what he or she should do ("Please select a data point.") after the "frmOpenEvent" was fired and the active (current) form was drawn.
Next, we handle the button press events. There are three buttons:

The clear button, which clears the field, in which the fetched data from the ESP32 will be displayed.
The bedroom button, which fetches the bedroom temp- and humidity data from the ESP32.
The basement button, which fetches the basement temp- and humidity data from the ESP32.

The clear button simply calls ClearField in UiAppLib.c, which replaces the content of the big text field with an empty string. The bedroom and basement buttons only sending a request to the ESP32 via the (serial) bluetooth interface. SendDataSerial is defined in SrAppLib.c. If the request was sent, we are waiting 10x 250ms for an answer from the ESP. If we get an answer, we process and display it with ProcessSerialData(rawData). If we do not get an answer within this time, we assume, that the connection got lost and displaying a alert form via handleConnectionLost in UiAppLib.c. The "exit" and "reconnect" functionality is handled inside of handleConnectionLost since we need this functionality at another place in the code, too. But we cannot be really sure, if the ESP32 doesn't take longer than the given time to process the request. So at this point the user has also the option to retry the request. We are simply waiting a little bit longer for an answer from the ESP32. If it takes too long again, we are displaying the same alert form again.
The stopApplication function actually only shuts down the Bluetooth stack. The getErrorExit and setErrorExit functions are used for leaving the application, when an error occurred. ProcessSerialData, like already mentioned, processes the incoming data from the ESP32 and display it. Since the data has always a fixed length, splitting the data can be done with fixed positions. It is not nice but it is working.

BtAppLib.c

#include <BtLib.h>
#include <BtLibTypes.h>
#include <BtCommVdrv.h>

#include "BluetoothComm.h"
#include "BtAppLib.h"

// returns the opened bluetooth port if needed
UInt16 getBtPort()
{
	UInt32 port = 0;

	FtrGet(appFileCreator, 9811, &port);
	
	return port;
}

// this starts the bluetooth stack as client
Err startBluetoothStack()
{
	Err returnCode = 0;
	UInt16 port = 0;
	
	SrmOpenConfigType config;
	BtVdOpenParams btParams;
	BtLibSdpUuidType sppUuid;
	
	MemSet( &sppUuid, sizeof(sppUuid), 0 );
	sppUuid.size = btLibUuidSize16;
	sppUuid.UUID[0] = 0x11;
	sppUuid.UUID[1] = 0x01;
	
	MemSet( &btParams, sizeof(btParams), 0 );
	btParams.role = btVdClient;
	btParams.u.client.method = btVdUseUuidList;
	btParams.u.client.u.uuidList.tab = &sppUuid;
	btParams.u.client.u.uuidList.len = 1;
	
	MemSet( &config, sizeof(config), 0 );
	config.function = 0;
	config.drvrDataP = (MemPtr)&btParams;
	config.drvrDataSize = sizeof(btParams);

	returnCode = SrmExtOpen(
			sysFileCVirtRfComm,
			&config,
			sizeof(config),
			&port
			);
			
	FtrSet(appFileCreator, 9811, port);

	return returnCode;
}

This BtAppLib.c library only contains two functions: getBtPort and startBluetoothStack. The first function uses the Palm OS Feature Manager (FtrGet) to provide the Bluetooth Port where ever it is needed. The second function (startBluetoothStack) collects all needed parameters to be able to start the bluetooth stack and also starts it. At the end of this funciton, the opened bluetooth port is then stored via the Feature Manager (FtrSet(appFileCreator, 9811, port);).

SrAppLib.c

#include "SrAppLib.h"
#include "BtAppLib.h"
#include "BluetoothComm_Rsc.h"
#include "BluetoothComm.h"
#include "BtAppLib.h"
#include "UiAppLib.h"

// This function reads the data, 
// which are received via bluetooth und returns it
Char* ReadSerial() {

  static Char buffer[100];
  static UInt16 index = 0;
  Err error;
  UInt32 bytes;
  UInt16 btPort;
  
  btPort = getBtPort();
  
  error = SrmReceiveCheck(btPort, & bytes);  
  if(error){
  
  	setErrorExit(error);
  
    ClearField(MainForm, DialogField);
	WriteInDialogField(MainForm, DialogField, "Error while receiving data!");

	handleConnectionLost(BtErrorOccurredAlert2);
	
	return NULL;
  }  
  
  if (bytes + index > sizeof(buffer)) {
    bytes = sizeof(buffer) - index -
      sizeOf7BitChar(\0);
  }

  while (bytes) {
    SrmReceive(btPort, & buffer[index], 1, 0, &error);
    if (error) {
      SrmReceiveFlush(btPort, 1);
      index = 0;
      return NULL;
    }
    switch (buffer[index]) {
    case chrCarriageReturn:
      buffer[index] = chrLineFeed;
    case chrLineFeed:
      buffer[index + 1] = chrNull;
      index = 0;
      return buffer;
      break;
    default:
      index++;
      break;
    }
    bytes--;
  }
  
  return NULL;
  
}


// This function sends text over the serial-(bluetooth)-interface
void SendDataSerial(void *value){

    Err error;
    UInt16 btPort;

	btPort = getBtPort();

    SrmReceiveFlush(btPort, 0);
    SrmSend(btPort, value, StrLen(value), &error);
    SrmSend(btPort, "\n", 1, &error);
        
    if(error){
    
    	setErrorExit(error);

    	ClearField(MainForm, DialogField);
    	WriteInDialogField(MainForm, DialogField, "Error while sending data!");
    
    	handleConnectionLost(BtErrorOccurredAlert2);
    }   
}

Also SrAppLib.c does only contain two functions, which are related to the serial communication. Bluetooth acts like a serial interface in Palm OS, so fetching and sending data is similar to a real serial interface on Palm OS. ReadSerial is for reading data from the port, when the ESP32 sends data to the PDA and SendDataSerial for sending data to the ESP32.

UiAppLib.c

#include "UiAppLib.h"
#include "BluetoothComm_Rsc.h"
#include "BtAppLib.h"
#include "BluetoothComm.h"

// This function handles a lost connection and offers are reconnect and exit
UInt16 handleConnectionLost(UInt16 alertID){
	
	UInt16 tappedButton;
	UInt16 btPort;
	
	btPort = getBtPort();
	tappedButton = FrmAlert(alertID);
	
	if(tappedButton == 0){
		//Reconnect
		ClearField(MainForm, DialogField);
		WriteInDialogField(MainForm, DialogField, "Restarting bluetooth stack...");
		SrmClose(btPort);
		startBluetoothStack();
		ClearField(MainForm, DialogField);
		WriteInDialogField(MainForm, DialogField, "Please select a data point.");
		setErrorExit(0);
	}
	
	if(tappedButton == 1){
		// Exit
		ClearField(MainForm, DialogField);
		WriteInDialogField(MainForm, DialogField, "Shutting down bluetooth stack...");
		setErrorExit(1);
	}
	
	return tappedButton;
}


// This function clears a field, primary used to clear the DialogField
Boolean ClearField(UInt16 formID, UInt16 fieldID)
{
	FormPtr 	formPtr;
	UInt16		objectIndex;
	FieldPtr	fieldPtr;
	UInt16 		length;
	Boolean 	handled = false;
	
	
	formPtr = FrmGetFormPtr(formID);
	objectIndex = FrmGetObjectIndex(formPtr, fieldID);
	fieldPtr = (FieldPtr)FrmGetObjectPtr(formPtr, objectIndex);
	length = FldGetTextLength(fieldPtr); 
	
	if(fieldID == DialogField){
		FldSetTextPtr(fieldPtr, "");
		FldDrawField(fieldPtr);
	}else{
		FldDelete(fieldPtr,0,length);
	}
	
    handled = true;
    
	return handled;
}

// This function writes a String into a (Dialog)Field
Boolean WriteInDialogField(UInt16 formID, UInt16 fieldID, MemPtr str)
{
	FormPtr 	formPtr;
	UInt16		objectIndex;
	FieldPtr	fieldPtr;
	
	char* dialogStrPtr;
	char* newDialogStrPtr;
	

	Boolean 	handled = false;
	
	formPtr = FrmGetFormPtr(formID);
	objectIndex = FrmGetObjectIndex(formPtr, fieldID);
	fieldPtr = (FieldPtr)FrmGetObjectPtr(formPtr, objectIndex);
	
	dialogStrPtr = FldGetTextPtr(fieldPtr);
	
	if(dialogStrPtr){
		newDialogStrPtr = MemPtrNew((StrLen(dialogStrPtr)+StrLen(str)+1));
	
		StrCopy(newDialogStrPtr, dialogStrPtr);
    	StrCat(newDialogStrPtr, str);
		FldSetTextPtr(fieldPtr, newDialogStrPtr);
		FldRecalculateField(fieldPtr, true);
	}else{
		FldSetTextPtr(fieldPtr, str);
	}
	
	FldDrawField(fieldPtr);
	
    handled = true;
    
	return handled;
}

UiAppLib.c handles the user interface actions like an alert from and the main display area for the data from the ESP32. The first function is called handleConnectionLost and handles a lost connection. If the connection lost is detected in MainFormHandleEvent (see BluetoothComm.c) or while sending (SendDataSerial) or receiving (ReadSerial) data (see SrAppLib.c), this function is called. It shows an alert form and provides two functions: "Reconect" and "Exit". If "Reconnect" is selected, the bluetooth connection gets shut down and started again. If "Exit" is selected "setErrorExit" will be set 1, which is being processed in AppEventLoop and will terminate the application if this value is 1.

Conclusion

The ESP8266 and ESP32 as well as the PHP Server are working very well. Only the Palm OS application is far away from perfect. But since this is a hobby project and not a professional one, it is good enough. Maybe I will find in the future some time for optimizations, like making the "request buttons" in the application dynamic by replacing them with a dropdown. So it would be possible to fetch all data points first and then showing them in the dropwdown list. It would be also nice to see in the main field from which room the data came. Or to have a proper error handling when the application gets started on a none-bluetooth-enabled device.
Besides from that, the Palm OS application has one known bug: When the device is turned off, while the application is running, it can crash or terminate with an alert, which is triggered in the PilotMain function. Also the ESP8266 will maybe also not return valid values every time. The last problem appears not very often. So I think both problems are acceptable.

At this point I want to thank the PalmDB community and especially Dmitry Grinberg, who helped me a lot with the implementation of the Palm OS application.

Bluetooth Communication with Palm OS - Processing and displaying the collected data (Part 2/2)