(c) 2024
Creating and processing with the Semantic Marker® System is through various APIs (Application Programer Interface). This GitHub repository includes code that makes web calls using the HTTP protocol and various languages, such as javascript, c and objective-c. The code included is mix of html and javascript. As these are a REST (HTTP) API, other languages make calls on these same web API's (such as the curl shell calls.) At the heart of the messaging API is the MQTT publish/subscribe engine described using the BNF Language grammar specification, and formatted through a collection of JSON messages sent over MQTT. Our Semantic Marker® ESP-32 IoT Device open-source downloadable software shows use of this MQTT messaging.
This document describes how the API supports the Semantic Marker® Architecture in the following sections:
The main interfaces to the SemanticMarker.org web services are through calls embodied in the following web page code SemanticMarker.org/smart. This web page (with html and javascript) is just one way to interface with the SemanticMarker API calls. Other web pages have been created that have a different user interface, while calling the same API's. In addition apps, like the Semantic Marker® iOS Apple App are in various phases of development, also interacting with the same API's.
Note
Many of these messages, after security credential matching, are then run through the MQTT publish/subscribe system described later.
These fit an overall cloud and device interaction Semantic Marker® architecture shown here and currently hosted at SemanticMarker.org.
Tip
The upper left of the diagram shows the versatility of these API's. Here additional clients use the API's to implement other features like voice recognition (e.g., Alexa, or Siri), and supporting other messaging systems like telegram, X, discourse, nodered, apache (tomcat) and slack.
The following are the main REST API calls to create and manage a Semantic Marker® . The current base URL for these is SemanticMarker.org but the entire set can also be run locally or moved to other web servers. For the Semantic Marker® to be sharable and run by outside parties, the Semantic Marker® address URL must be publically available (for example, https://SemanticMarker.org is globally accessible, while http://localhost would only be available for local use - such as a dog kennel operation.)
The list of API calls is shown next with full details later in their appropriate sections.
Table of REST, MQTT and BLE API Calls
name description parameters protocol /exists/ks query if Semantic Marker® exists {category}/{namespace} /{UUID}/{escapedSemanticMarker} GET /train/ks train a Semantic Marker® {namespace}/{category} /{UUID}/{escapedSemanticMarker} GET /sm create a new Semantic Marker® {namespace}/{category} /{UUID}/digitize POST /smrun Runs a Semantic Marker® parameters GET /smflowpost Create or modify Semantic Marker® JSON format POST /invokeSemanticMarker Invokes a Semantic Marker® {username}/{password} /{SemanticMarkerAddress} GET /createSemanticMarker Create a Semantic Marker® {username}/{password} /{SemanticMarkerAddress}/{AvatarURL} GET /feed/Send feed command {username} {password} GET /feeddevice Send feed to specific named device {username}/{password} /{devicename} GET /lookupuser Lookup user returning JSON {username}/{password} GET /set sends a set/val {username}/{password}/{command}/{value} GET /setdevice sends a set/val to device {username}/{password} /{devicename}/{command}/{value} GET /send sends a request {username}/{password}/{request} GET /senddevice sends a request to a device {username}/{password} /{device}/{request} GET /set64device sends command in base64 {username}/{password} /{device}/{command}/{base64Val} GET /command send boolean command {username}/{password} /{command}/{on/off} GET /cmddevice send boolean command to device {username}/{password} /{device}/{command}/{on/off} GET /cmd send command to device {username}/{password} /{device}/{command} GET /addGroup add to group {username}/{password}/{group} GET /removeGroup remove from group {username}/{password}/{group} GET /statusGroup send status to group {username}/{password} /{group} GET /feedGroup send feed to group {username}/{password} /{group} GET /feedDeviceGroup send feed to group and device {username}/{password} /{device}/{group} GET /captureGroup sends capture to group {username}/{password}/{group} GET /addDeviceToGroup adds device to group {username}/{password} /{device}/{display}/{color}/{group} GET /addUserToGroupTopic add user to group {group}/{username}/{TOKEN} GET /addUser creates user account {username}/{password}/{guestpassword}/{TOKEN} GET mosquitto_pub publish on topic -i userID -u username -P password -t topic -m PAYLOAD MQTT mosquitto_sub subscribe to topic -i userID -u username -P password -t topic MQTT addMosquitoUser Add guest user to group user password guestPassword MQTT addMosquitoGroup Add user to group user group MQTT /connectBLE Binds to the BLE device service characteristics BLE /sendCommandBLE Sends character over BLE character BLE /sendJSONCommandBLE Sends JSON over BLE JSON BLE /scannedDevice Denotes the device scanned JSON MQTT
Tier II and Tier III use the Semantic Marker® Registry (ie. database) requiring a user account. This is anchored by a unique UUID for that user.
In addition to valuable IoT message addresses contained in the Semantic Marker® - we add a user specified graphic photo image, your Photo Avatar, embedded inside. This helps the user distinguish among potentially 1000's of these new optical vision markers (maybe stored in your photo album). The human eye can pick out different Avatars faster than decoding with our Semantic Marker® scanners.
This Photo Avatar is an important feature of Semantic Marker® and shown as this process:
Combining the Optical Marker with the Visual Marker (the Avatar) creates a unique combination of capability and semantic meaning we trademarked into the Semantic Marker® - an Optical Visual Marker.
Tip
We humans (mostly) use our Visual skills to make sense of our environment. But we also use Optical tools to enhance (or even make sense) of the same visual environemnt. For example, an optical telescope brings things closer for better understanding (like Planets or climbers on a wall.) Vision recognition of faces or text is also useful. The advantage of a visual two-dimensional code is that special optics can now translate the image into perfect information. So our vision is extended with optics resulting in an Optical Visual Marker - or our Trademarked name: Semantic Marker®
Note
Throughout this API document, a black arrow 
denotes expandable details. Clicking will expand that section
and show more information (such as the parameter details when using the API.)
A Tier I Semantic Marker® is a straight indirection to a remote address. These are created directly from the various language implementation. Examples of javascript, C and objective-c are shown. These are snipits of code are from various Semantic Marker® apps to give a flavor of the APIs.
A valuable use of Tier I is a Photo Bookmark that links to photo cloud storage or social media stories. Basically the metadata, the hyper-links, are stored in the Photo Bookmark, our Semantic Marker®. They can be printed out, handed out, stored in on-line photo albums, and they all expose their hyper-links via any newer smart phone scanner.
Web page creating Semantic Marker®
The following web page image is the result of the current Tier I Semantic Marker® service running at SemanticMarker.org - under the "Use our Service to Create your own Semantic Marker®" tab. Clicking "Create Semantic Marker®" will create the image below (with potentially another Graphic Avatar - or user selected.)
Another example html web page with many javascript calls via these REST API's is included at: smart.html
Example production web pages includes making appropriate API calls includes: iDogWatch.com and SemanticMarker.org
In addition, applications such as we developed for the ESP-32 family of embedded IoT devices, show use of this API. See Semantic Marker ESP-32 Download for an example.
ESP-32 M5 Code create Semantic Marker
This is a simple example of creating a Tier I Semantic Marker® inside the ESP-32 code base. Ths code is available via our open-source software at Semantic Marker® ESP-32.
//!Draws a SemanticMarker®
//!Create a SemanticMarker code with a width of 135 QR code at (0, 100).
//!Please select the appropriate QR code version according to the number of characters.
createSemanticMarker("https://iDogWatch.com/bot/help",0,100,135);
//try to draw a character in the middle ...
printTextAtTextPosition("SM",_zoomedTextPositions[QRAVATAR_ACTION]);
Apple iOS Objective-C create Semantic Marker
This is a simple example of creating a Tier I Semantic Marker® inside the iOS Apple Objective-c code base.
The createSemanticMarker is a method that performs all the Semantic Marker® image merging and creation. The image merging is placing the avatarImage inside the resulting Semantic Marker® at a size and aspect ratio inherent to the Semantic Marker® specifications.
//! create new image with the Avatar Logo inside
UIImage *image = [self createSemanticMarker:address avatarImage:image];
Note this image captured from the Semantic Marker® Apple Watch, shows the Semantic Marker® (where a long press will invoke the Semantic Marker® using the Tier III REST API), but also shows the FEED button. This makes a call on the /feed REST API shown below.
Javascript new SemanticMarker
The SMART web page makes calls through various javascript http calls (shown as XMLHttpRequest below). These are embedded in the smart.js code modules.
Excerpt from html file:
//The commandPath (the Semantic Marker® address is a parameter as is the avatarURL, and if the
//avatar is circular or rectangle).
function createSemanticMarker(commandPath, avatarURL, circular)
{
try
{
// http web call to get the image from the AvatarURL
getImage(avatarURL).then
(
function (successurl)
{
var sm = new SemanticMarker(document.getElementById(idSM),
commandPath,
avatarURL,
circular);
}).catch (function (errorurl)
{
alert("Issue creating Avatar in Semantic Marker® creation: \n" + errorurl + "\nPlease file bug report");
})
}
catch (error)
{
alert("Issue with Semantic Marker® creation: \n" + error.message + "\nPlease file bug report");
}
}A typical Http web call looks like the following, passing in a command which includes calls to the appropriate web interface (for example https://SemanticMarker.org/bot/createSemanticMarker as shown in the diagram above).
//! sends the GET command to create a Semantic Marker
function createSemanticMarker(command)
{
var xhttp = new XMLHttpRequest();
//!https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/readystatechange_event
xhttp.onreadystatechange = function() {
// called when open is finished
if (this.readyState == 4 && this.status == 200)
{
alert("SemanticMarker was created");
}
};
xhttp.open("GET", command, true);
xhttp.send();
} A Tier II Semantic Marker® invoves storing information at one of our Semantic Marker® cloud servers. This includes an indirect address that a valid user can modify. Later, the Tier III Semantic Marker® provides full user control. Both Tier II and Tier III require a user account since the Semantic Marker® is stored in a database.
GET /exists/ks/{category}/{namespace}/{UUID}/{escapedSemanticMarker}
name type data type description namespace required string Namespace of Semantic Marker category required string Category of Semantic Marker UUID required string UUID of the user escapedAddress required string The Semantic Marker Address is escaped so it is a single parameter argument
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/exists/ks" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
GET /train/ks/{namespace}/{category}/{UUID}/{escapedSemanticMarker}
name type data type description namespace required string Namespace of Semantic Marker category required string Namespace of Semantic Marker UUID required string UUID of the user escapedAddress required string The Semantic Marker Address is escaped so it is a single parameter argument
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/train/ks" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
A Tier III Semantic Marker® provides storage for the Avatar image and for very fine control of Semantic Marker actions. The created result is called a SMART Button - standing for Semantic Marker® Augmented Reality of Things. With knowledgable apps, the SMART Buttons can extract the Photo Avatar and use as a Augmented Reality overlay.
Tip
The resulting stored processing steps are especially suited to customized Internet of Things (IoT) processing, such as a custom user manual for your IoT home devices (eg. your accounts, passwords, WIFI address, and device Id's). These utilize all the API calls shown in this document but customized to your environment..
A SMART Button has a unique Semantic Marker® address such as:
POST /sm/{namespace}/{category}/{UUID}/digitize
name type data type description namespace required string Namespace of Semantic Marker category required string Namespace of Semantic Marker UUID required string UUID of the user
http code content-type response 201text/plain;charset=UTF-8http URL of resulting Avatar Image400application/json{"code":"400","message":"Bad Request"}
-
Image file to be digitized
-
JSON of credentials
Like many of these interfaces, the credentials including the username and password, are passed as part of the payload in these POST calls.
{
"username":_username,
"password":_password,
"name" : flowName,
"uuid" : uuid,
"flowNum" : flowNum,
"flowCat" : flowCat
}set fullsm = "https://SemanticMarker.org/bot/sm/$name/$cat/$uuid" curl --trace-ascii curl.trace \ -F filename=$filename \ -F upload=@$filename \ -F username=$user \ -F password=$pass \ -F link=$link \ -F uuid=$uuid \ -F flownum=$flownum \ -F flowcat=$flowcat \ -F kind=$kind \ $fullsm
POST /smflowpost
name type data type description
http code content-type response 201text/plain;charset=UTF-8http URL of resulting Avatar Image400application/json{"code":"400","message":"Your SemanticMarker Address was already registered."}
- JSON of credentials for the SMART Button
Like many of these interfaces, the credentials including the username and password, are passed as part of the payload in these POST calls.
{
"username":_username,
"password":_password,
"name" : flowName,
"uuid" : uuid,
"isdata": isDataFlag,
"private": privateflag,
"flowNum" : flowNum,
"flowCat" : flowCat,
"date" : dateString,
"parameters" : parameters,
"desc": desc,
"QRAvatarURL":QRAvatarURL,
"nextFlowURL":nextFlowURL,
"flow" : allCommandsJSON,
"dataSM" : dataSM,
"KSMatrix" : KSMatrixText,
"artifactsSM" : artifactsSM,
"markup" : markupText,
"languagesSM": languagesSM,
"videoSM": videoSM,
"isCircularAvatar": isCircularAvatar,
"KSWave" : KSWaveText,
"bridgeSM" : bridgeSM,
"future" : futureText,
"audioSM" : audioSM,
"locationSM" : locationSM,
"inheritedSM" : inheritedSM,
"deckSM" : deckSM,
"isMessaging" : isMessaging,
"isIndirectSM" : isIndirectSM,
"indirectSM" : indirectSM
}set fullsm = "https://SemanticMarker.org/bot/smflowpost" curl --trace-ascii curl.trace \ $fullsm
GET /smrun
name type data type description uuid string string uuid of SMART button flow string string flownum of SMART button username string string username password string string password device string string device others.. string string other parameters
http code content-type response 201text/plain;charset=UTF-8http URL of resulting Avatar Image400application/json{"code":"400","message":"Your SemanticMarker Address was already registered."}
set fullsm = "https://SemanticMarker.org/bot/smrun?uuid=x&flow=y&username=X&password=Y&device=Z" curl --trace-ascii curl.trace \ $fullsm
Here, there are 2 flow steps, with an implicit if (device) do the first, otherwise do the second. https://iDogWatch.com/bot/cmddevice/USERNAME/PASSWORD/DEVICE_1/feed https://iDogWatch.com/bot/cmd/USERNAME/PASSWORD/feed
Caution
SMART Buttons can be run by anyone but the internal IoT messages only work with valid user credentials (username, password). The results will be calls to the other Semantic Marker API's after substituting those query parameters. Also with inheritance the same SMART Button can include extended user specific values. Inheritance requires a user unique uuid but the same flow.
The results of these REST calls are secure JSON database entries in the Semantic Marker® registry, as shown in this image.
These are the Websocket and MQTT messaging formats. This includes the original iDogWatch websocket interface defined as the Barklet Language. The format is a mix of original Barklet Language used with WebSockets, and the newer JSON Format messages. Example uses are available via our open-source at Semantic Marker® ESP-32.
Note
Many of the messages described in the following BNF Grammar can be run through the UNIX shell commands at: Example curl commands To run, download the scripts and run with a valid username and password. CURL is a tool to send web REST commands from the command line.
Description ::= Grammar for Barklet communication
NOTE ::= {} are part of language not BNF
NOTE ::= : <guest ID> ":" created by chat-room, not user messages
message ::= [<guest ID> ":"] <payload> <player_name> [<time>]
| '{' [<device>] <JSONMessage> '}'
payload ::= <request> [<deviceInfo>] [<SemanticMarker>] [<OTA>]| <reply>
request ::= #STATUS | #TEMP | #CAPTURE | #FEED
| #VERSION | #DOCFOLLOW | #followMe | #CLEAN_SSID_EPROM | #OTA
reply ::= <connection>
| <location>
| <biometrics> <bot_name>
| <ack>
| <chat>
| <version>
| <distanceTo< <bot_name>
SemanticMarker ::= AVM= <SMAddress>
SMAddress ::= SemanticMarkerURL
OTA ::= {v=Version} | {k=<kind>}
kind ::= ESP_M5 | ESP_32
connection ::= <connectionStatus> <status>
connectionStatus ::= <connectionPhase> <bot_name> | #REMOTE
connectionPhase ::= #CONNECTED | #WAITING | #DISCONNECTED
status ::= {I,F,remote} //Inside network, Foreground || Outside, background
location ::= #GPS <loc>
loc ::= {nil} | {<lat:float>,<lon:float>,<alt-feet:float>}
chat ::= #CHAT_CONNECTED | #CHAT_DISCONNECTED
ack ::= #ACK <ack_kind>
ack_kind ::= <ack_feed> | <docfollow>
ack_feed ::= "Fed" <bot_name>
biometrics ::= <temp> | <acceleration>
temp ::= {TempF=<float>}
acceleration ::= {Acc=<floatX>,<floatY>,<floatZ>}
deviceInfo ::= <identity> | {"deviceName":<name>}
bot_name ::= <identity>
player_name ::= <identity>
identity ::= {<name> [<UUID>]}
UUID ::= <32 bit name>
float ::= <int> : <int>
time ::= {hh:mm:ss}
version ::= {ver=x.x}
number ::= <int>
boolean ::= "on" | "off"
JSONMessage ::= <set> | <setdevice> | <send> | <set64> | <SemanticMarkerApp Messages>
device ::= "device" : name_of_device
setdevice ::= <device> <set>
set ::= "set" : setString , "val": valString
send ::= "send" : sendString
set64 ::= "set64" : <encodedBase64String>
setString ::= "semanticMarker"
| "highTemp" <number>
| "feed" <boolean>
| "status" <boolean>
| "resetTimer" <boolean>
| "devOnlySM" <boolean>
| "ble+wifi" <boolean>
| "factoryReset" <boolean>
| "restartModels" <boolean>
| "screenTimeout" <number>
| "stepperAngle" <number>
| "noclick" <boolean>
| "gateway" <boolean>
| "DiscoverM5PTClicker" <boolean>
| "useSPIFF" <boolean>
| "useSPIFF_mqtt" <boolean>
| "useSPIFF_qratom" <boolean>
| "readspiff"
| "deletespiff"
| "sendspiff"
| "timerDelay" <number>
| "timerDelayMax" <number>
| "startTimer" <boolean>
| "stepper" <number>
| "clockWiseMotor" <boolean>
| "otaFile" <string>
| "location" <string>
| "device" <string>
| "pairNow" <boolean>
| "pairDev" <string>
| "screenColor" <number>
| "gen3only" <boolean>
| "BLEUsePairedDeviceName" <boolean>
| "BLEUseDeviceName" <boolean>
| "minMenu" <boolean>
| "addWIFI" ssid:password
| "useDOCFOLLOW" <boolean>
| "semanticMarker"
| "blankscreen" <boolean>
| "SubDawgpack" <boolean>
| "buzz" <boolean>
| "BLEClient" <boolean>
| "BLEServer" <boolean>
| "tilt" <boolean>
| "zoomSM" <SMNum>
| "buttonA" ["longpress" | "shortpress"]
| "buttonB" ["longpress" | "shortpress"]
| "scannedDevice" <string> | "_none"
| "scannedGroup" <string> | "_none"
| "socket" <boolean>
| "atomscanner" <boolean>
| "PIR_UseSM" <boolean>
| "AtomSocketGlobalOnOff" <boolean>
| "LUXdark" <int eg. 80>
| "M5AtomKind" <M5Scanner=0, M5_Socket=1>
(more as of 8.15.2024)
| "useGroups" <boolean>
| "groups" <list of groups: atlasDogs, houndDogs) or (#) or "") -- Deprecated
| "includeGroups" <comma seperated group names>
| "stepperangle" <angle number>
| "autoMotorDirection" <boolean>
sendString ::= "temp"
| "status"
| "capture"
| "volume"
| "feed"
| "togglesocket"
encodedBase64String ::=
| <Semantic Marker value after decoding base64>
| "semanticMarker" <val>
| "PIR_SM_JSON" <val>
| "PIR_SM_OFF_JSON" <val>
SemanticMarkerAppMessages ::=
| DEBUG <boolean>
| HUD <boolean>
| MQTT_DEBUG <boolean>
| SMFocusCapture <boolean>
| SMSharing <boolean>
| SMSharingLeader <boolean>
| SM_AR_Minimal <boolean>
| SM_AR_ZOOM <boolean>
| SM_AR_Zoom <boolean>
| SM_Flip <boolean>
| SM_FlipRotate <boolean>
| SM_Instance_Image <boolean>
| SM_QRAvatar <boolean>
| SM_ReplaceUUID <boolean>
| SM_UseCropImage <boolean>
| SM_VisionLinker <boolean>
| SM_ShowMatrix <boolean>
| SM_FocusPause <boolean>
| SM_SaveScanned <boolean>
| SM_ShowLabels <boolean>
| SM_ShowMatrix <boolean>
| SM_UsePhotoLocation <boolean>
| SM_SparseMatrixVisionLinker <boolean>
| SM_Glasses <boolean>
| animals <boolean>
| images <boolean>
| matrix <boolean>
| noclick <boolean>
| pause <boolean>
| pdf <boolean>
| replaceUUID <UUID string>
NEW: 8.9.24 (Jerry Garcia memorial day)
| SM_PageTurner <boolean>
| SM_DOCFOLLOW <boolean>
| SM_DOCFOLLOW_Leader <boolean>Note
In addition to the REST syntax, URL query parameters are sometimes used. For example, supplying query parameters and their values for username, password and with a JSON format reply would look as follows: https://SemanticMarker.org/smart?username=x&password=y&format=JSON
GET /invokeSemanticMarker/{username}/{password}/{SemanticMarkerAddress}
name type data type description username required string username of user password required string password of user semantic marker address required string semantic marker address
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/invokeSemanticMarker/USERNAME/PASSWORD/SEMANTIC_MARKER" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
GET /createSemanticMarker/{username}/{password}/{SemanticMarkerAddress}/{AvatarURL}
name type data type description username required string username of user password required string password of user semantic marker address required string semantic marker address avatar URL required string image avatar URL
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/createSemanticMarker/USERNAME/PASSWORD/SEMANTIC_MARKER/AVATAR_URL" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
GET /feed/{username}/{password}
name type data type description username required string username of user password required string password of user
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/feed/USERNAME/PASSWORD" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
GET /feeddevice/{username}/{password}/{devicename}
name type data type description username required string username of user password required string password of user deviceName required string name of device
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/feeddevice/USERNAME/PASSWORD/DEVICE" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
GET /lookupuser/{username}/{password}
name type data type description username required string username of user password required string password of user
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/lookupuser/USERNAME/PASSWORD" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
{
"username":_username,
"password":_password,
"name" : flowName,
"uuid" : uuid,
"isdata": isDataFlag,
"private": privateflag,
"flowNum" : flowNum,
"flowCat" : flowCat,
"date" : dateString,
"parameters" : parameters,
"desc": desc,
"QRAvatarURL":QRAvatarURL,
"nextFlowURL":nextFlowURL,
"flow" : allCommandsJSON,
"dataSM" : dataSM,
"KSMatrix" : KSMatrixText,
"artifactsSM" : artifactsSM,
"markup" : markupText,
"languagesSM": languagesSM,
"videoSM": videoSM,
"isCircularAvatar": isCircularAvatar,
"KSWave" : KSWaveText,
"bridgeSM" : bridgeSM,
"future" : futureText,
"audioSM" : audioSM,
"locationSM" : locationSM,
"inheritedSM" : inheritedSM,
"deckSM" : deckSM,
"isMessaging" : isMessaging,
"isIndirectSM" : isIndirectSM,
"indirectSM" : indirectSM
}GET /set/{username}/{password}/{command}/{value}
set ::= "set" : setString , "val": valString
name type data type description username required string username of user password required string password of user command required string command for value required string value for
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/set/USERNAME/PASSWORD/COMMAND/VAL" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
GET /setdevice/{username}/{password}/{devicename}/{command}/{value}
setDevice ::= "device": device, "set" : setString , "val": valString
name type data type description username required string username of user password required string password of user deviceName required string name of device command required string command for value required string value for
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/setdevice/USERNAME/PASSWORD/DEVICE/COMMAND/VAL" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
GET /send/{username}/{password}/{request}
send ::= "send" : sendString
name type data type description username required string username of user password required string password of user request required string command request string
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/send/USERNAME/PASSWORD/DEVICE" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
GET /senddevice/{username}/{password}/{device}/{request}
sendDevice ::= "device": device, "send" : sendString
name type data type description username required string username of user password required string password of user device required string device to receive message request required string command request string
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/senddevice/USERNAME/PASSWORD/DEVICE/REQUEST" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
GET /set64device/{username}/{password}/{device}/{command}/{base64Val}
set64device ::= "device": device, "set64" : <encodedBase64String>
name type data type description username required string username of user password required string password of user device required string device to receive message command required string command request string base64val required string the value but encoded in base64
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/set64device/USERNAME/PASSWORD/DEVICE/BASE64VAL" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
GET /command/{username}/{password}/{command}/{on/off}
command ::= "command" : command, "val": on/off
name type data type description username required string username of user password required string password of user command required string command request string val required string the value in boolean (on/off)
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/command/USERNAME/PASSWORD/COMMAND/BOOLEAN_VAL" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
cmddevice - sends boolean value to the specified device, basically a request but the value is on/off
GET /cmddevice/{username}/{password}/{device}/{command}/{on/off}
cmdDevice ::= "device": device, "command" : command, "val": on/off
name type data type description username required string username of user password required string password of user device required string device to receive message command required string command request string val required string the value in boolean (on/off)
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/cmddevice/USERNAME/PASSWORD/DEVICE/COMMAND/BOOLEAN_VAL" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
GET /cmd/{username}/{password}/{device}/{command}
cmd ::= "device": device, "command" : command
name type data type description username required string username of user password required string password of user device required string device to receive message command required string command request string val required string the value in boolean (on/off)
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/cmddevice/USERNAME/PASSWORD/DEVICE/COMMAND/BOOLEAN_VAL" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
The following subset of REST calls are for managing Group entry for the MQTT Publish/Subscribe capabilies.
GET /addGroup/{username}/{password}/{group}
name type data type description username required string username of user password required string password of user group required string group name
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/addGroup/USERNAME/PASSWORD/GROUP" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
GET /removeGroup/{username}/{password}/{group}
name type data type description username required string username of user password required string password of user group required string group name
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/removeGroup/USERNAME/PASSWORD/GROUP" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
GET /statusGroup/{username}/{password}/{group}
name type data type description username required string username of user password required string password of user group required string group name
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/statusGroup/USERNAME/PASSWORD/GROUP" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
GET /feedGroup/{username}/{password}/{group}
name type data type description username required string username of user password required string password of user group required string group name
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/feedGroup/USERNAME/PASSWORD/GROUP" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
GET /feedDeviceGroup/{username}/{password}/{device}/{group}
name type data type description username required string username of user password required string password of user device required string device name group required string group name
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/feedDeviceGroup/USERNAME/PASSWORD/DEVICE/GROUP" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
GET /captureGroup/{username}/{password}/{group}
name type data type description username required string username of user password required string password of user group required string group name
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/captureGroup/USERNAME/PASSWORD/GROUP" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
GET /addDeviceToGroup/{username}/{password}/{device}/{display}/{color}/{group}
name type data type description username required string username of user password required string password of user device required string name of your device display required string name you want to call your device (eg. GroupA_DeviceB) color required string color of button (not implemented yet) group required string group name
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/captureGroup/USERNAME/PASSWORD/DEVICE/DISPLAY/COLOR/GROUP" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
Caution
This requires a secure TOKEN password managed by the system administrators
GET /addUserToGroupTopic/{group}/{username}/{TOKEN}
name type data type description group required string group name username required string username of user token required string token for admin to use
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/addUserToGroupTopic/GROUP/USERNAME/TOKEN" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
Caution
This requires a secure TOKEN password managed by the system administrators
GET /addUser/{username}/{password}/{guestpassword}/{TOKEN}
name type data type description username required string username of user password required string password of user guestpassword required string guestpassword of user token required string token for admin to use
http code content-type response 201text/plain;charset=UTF-8Configuration created successfully400application/json{"code":"400","message":"Bad Request"}
set fullsm = "http://localhost:1880/addUser/USERNAME/PASSWORD/GUESTPASSWORD/TOKEN" curl -v -F username=$user -F password=$pass -F link=$link -F kind=$kind $fullsm
In addition to the REST Web calls, the lower level MQTT messaging can be used directly. This requires that the MQTT pub/sub engine be available. Semantic Marker® currently uses the Eclipse Mosquitto® open source MQTT broker. This is available at mosquitto.org.
The dPub is a sample of how a valid user could publish a payload on a topic
mosquito_pub mosquitto_pub -i userID -u username -P password -t topic -m PAYLOAD
name type data type description userID required string unique user ID (for each publish) username required string username of user password required string password of user topic required string command request string PAYLOAD required string the payload to send (if on shell this must be escaped)
The dSub is a sample of how a valid user could subscribe to a valid topic. The generic topic is usersP/bark/#.
mosquito_sub mosquitto_sub -i userID -u username -P password -t topic
name type data type description userID required string unique user ID (for each publish) username required string username password required string password of user topic required string command request string
The MQTT pub/sub engine uses topics and permissions to constrain who can receive and who can send information (payload) through the network. The following diagram shows the MQTT Topic Namespace, including a hierarchy of permissions. For example, everyone can communicate with others in the same account, but not across acounts. There is a super account that can send messages to everyone. In addition, there is a dawgpack topic that everyone can collaborate on (much like an old party line). So a status could be requested on the super topic, and replies sent to the dawgpack. In addition, group topics are another way to communication across user accounts (by subscribing to group topics).
The following diagram highlights this namespace design.
Note
the power of this Namespace Design is that new topics, especially groups, can be created without modifying existing subscriptions. This is especially powerful for the embedded ESP-32 IoT devices without modification as they will automatically receive messages as controlled by the MQTT messaging broker. Membership in groups is performed through the REST calls and the following User and Group Commands.
The shell commands for creating users and topics include the following.
Caution
These shell commands must be run on the same computer that can access the user and password files. These commands should only be run by an approved user (and the REST API requires a special TOKEN or password that should be guarded).
addMosquitoUser addMosquitoUser user password guestPassword
Shell command: addMosquitoUser.sh
name type data type description user required string username password required string password of user guestPassword required string guest password
addUserToGroup addMosquitoUser user group
Shell command: addUserToGroup.sh
name type data type description user required string username group required string group name
These consist of single character commands (so bluetooth clients with length restrictions can send commands), and JSON formatted messages which is a subset of JSON messages sent over MQTT. Example uses are available via our Semantic Marker® ESP-32 open-source download for the BLE Client and BLE Server.
Caution
The ESP devices have a small buffer for accepting JSON messages (about 500 characters) so sending these JSON messages will be cut off if the message is too long (an error as not a valid JSON format.) Also there is currently no security with BLE as anyone that knows the protocol can connect and make updates.
BLE Bluetooth ESP-32 Device Interfaces
Must Discovery and Bind to desired device.
/connectBLE connect to BLE service
name value description SERVICE_UUID B0E6A4BF-CCCC-FFFF-330C-0000000000F0 BLE Service UUID CHARACTERISTIC_UUID B0E6A4BF-CCCC-FFFF-330C-0000000000F1 BLE Characteristic UUID
/sendCommandBLE Sends Command Message over BLE
value description . help W restartWIFI w swap WIFI N sendWIFI n nextWIFI a auto feed ON A auto feed OFF j jackpot B buzzerOn b buzzerOff s single feed 0x00 single feed c feed u uno m mini L tumbler G gatewayOn g gatewayOff R clean credentials - AP Node X clean EPROM - Factory Reset E use only PTFeeder naming e use naming PTFeeder:name T tiltOn t tiltOff O otaUpdate r reboot H autoMotorDirection on h autoMotorDirection off
Note
Topic Support will be Super, Device, Own, Group, DawgPack
Meaning the message is processed if that topic kind is supported and/or the
device name (or query) matches the processing device. eg. ODG says the device must be specified, but supports messages from a group, and it's own topic.
/sendJSONCommandBLE Sends Command Message in JSON format over BLE
JSON value description Topic Support SDOGP {'cmd':'feed'} perform a feed SOG {'cmd':'status'} status SOG {'cmd':'wifi'} update WIFI O {'ssid':'SSID','ssidPassword':'PASS'} set SSID and SSID Password O {'username':'NAME','password':'PASS'} set username and Password for MQTT meessaging O {'cmd':'sm1'} M5 change to the s1 (or other) pages OG {'cmd':'clean'} clean the EPROM O {'cmd':'erase'} erase the EPROM O {'cmd':'poweroff'} turn the power off on the device O {'cmd':'ota'} perform an OTA - Over the Air update O {'cmd':'buzzon'} turn buzzer on OG {'cmd':'buzzoff'} turn buzzer off OG {'cmd':'bleclienton'} turn the BLE client feature on OD {'cmd':'bleclientoff'} turn the BLE client feature off OD {'cmd':'bleserveron'} turn the BLE Server on OD {'cmd':'bleserveroff' turn the BLE Server off OD {'cmd':'resetfirsttime'} reset the first time flag OD {'cmd':'reboot' reboot the IoT device OD {'set':'BLEUseDeviceName','val':'on'"} use the Device Name for the BLE name service name OD {'set':'BLEUseDeviceName','val':'off'"} Don't use the Device Name for the BLE name service name OD {'set':'pairnow','val':'on'"} have the device pair to the named pair device OD {'set':'timerdelay','val':'30'"} set the timer delay to the val specified (eg. 30 sec) OD {'set':'startTimer','val':'on'"} start the timer OD {"startTimer" val:@"off"} stop the timer OD {'cmd':'zoomSMOn'} M5 display zooms and shows the Semantic Marker® OG {'cmd':'zoomSMoff'} Don't show the Semantic Marker® OD {'set':'tilt','val':'on'} turn tilt detection on OD {'set':'tilt','val':'off'} turn tilt detection off OD {'set':'gateway','val':'on'} turn the gateway value on OD {'set':'gateway','val':'off'} turn the gateway value off OD {'set':'hightemp','val':'88'} set the high temp detector to the val OD {'set':'stepperangle','val':'30'} change the stepper angle OD {'set':'screentimeout','val':'400'} change the screen timeout OD {'set':'autoMotorDirection','val':'on/off'} motor will change directions each feed OD {'set':'noclick','val':'400'} change if no click since val to turn off OG {'set':'stepper','val':'mini'} change the kind of stepper OD {'set':'stepper','val':'uno'} change the kind of stepper OD {'set':'stepper','val':'tumbler'} change the kind of stepper OD {'set':'device','val':'MyFeederName'} name the device used in messages and BLE Device OD {'set':'location','val':'Buckley, WA USA'} specify location to city/state/country OD {'set':'gen3only','val':'true'} turn on gen3 pairing only O {'set':'gen3only','val':'false'} turn off gen3 pairing only O {'send':'temp'} sends the temperature of the device SOG {'send':'capture'} captures an image and sends it SOG {'send':'volume'} specify the volume to be sent OG
As mentioned elsewhere, the group concept is powerful. After multiple users are members of the same group,
most messages to those groups are processed by everyone listening (unless specific device level command.)
But for security, some devices will opt out of specific shared groups. Thus the addition of a new state variable and
associated messages: includeGroups.
/sendJSONCommandBLE Sends Group Command Message in JSON format- WIFI or BLE
JSON value description Group Support SDOGP {'set':'includeGroups','val':'comma list'"} Groups to listen for, comma seperated OD
/sendJSONCommandBLE Sends ATOM Command Message in JSON format over BLE
JSON value description Topic Support SDOGP {'set':'scannedDevice','val':'deviceName'} temp sets the device name DO {'set':'scannedGroup','val':'groupName'} temp sets the publishe group name DO {'set':'atomSocketGlobalOnOff','val':'boolean'} socket accepts on/off from groups DO {'set':'socket','val':'boolean'} turns on/off the M5 Atom Socket O & optionally DG {'send':'toggleSocket'} toggles on/off the M5 Atom Socket DO optionally G {'set':'atomScanner','val':'boolean'} turns on/off the scanner (not yet) DO {'set':'LUXdark','val':'int'} specify when DARK starts for LUX (eg 80) DO {'set':'PIR_UseSM','val':'boolean'} motion or light sensor call SM, otherwise FEED DO {'set64':'semanticMarker','val':'base64 JSON'} Sets the SM command call DO {'set64':'PIR_SM_JSON','val':'base64 JSON'} Sets the SM command for the sensor to call when on DO {'set64':'PIR_SM_OFF_JSON','val':'base64 JSON'} Sets the SM command for the sensor to call when off DO
Outside of creating a Semantic Marker®, modification and invocation is out of scope of this API document. One web page that encapsulates most of these interactions can seen at: SemanticMarker.org/smart
The Semantic Marker® supports existing and future two-dimensional optical visual markers. The following are a couple examples.
The resulting Semantic Marker® can look like the following examples.
This Semantic Marker® encoded sensor status in it's address. For this, the M5
is constantly changing this address to reflect changing sensor status (like battery amount
or feed count). This is similar to how newer tickets to concerts are handled where the marker
constantly changes to invalidate a copied screen image of anothers ticket.
This Semantic Marker® uses the /smrun API to pass parameters (like username, password and device) so the stored procedures work securely across any user. The query parameters uuid and flow denote this SMART Button instance.
See PAT
Note
Last updated 8.16.24 Last updated 5.25.24 Last updated 4.14.24 Last updated 1.18.24
Eye Tracker capable for speech and sending messages. The Matrix collects these nicely.
This is the feature where a set of SMART buttons make up a matrix that can be shown on a web page, or the SemanticMarker iOS iPhone/iPad app.
The format of the matrix is a simple text file in the future of the agent flow definition.
@matrix(1,1,Matrix,Sit,QHmwUurxC3,1713138730023,--ALL--
1,2,ESP32Feeder,FeedDog,QHmwUurxC3,12,--ALL--
2,2,M5Category,PairDevicePage,QHmwUurxC3,100,--ALL--
2,3,M5Category,BLE_WIFI_Feed,QHmwUurxC3,1685637621385,--ALL--
3,2,M5Category,WIFI_feed,QHmwUurxC3,1674172214476,--ALL--
3,3,Matrix,FeedMatrix,QHmwUurxC3,1713138245068,--ALL--
)
Semantic Marker SMART Matrix Definition
row col SMART Category Smart Name UUID , FLOWNum Device 1 1 Matrix Sit QHmwUurxC3,1713138730023 --ALL-- 1 2 ESP32Feeder FeedDog QHmwUurxC3,12 --ALL--
A Knowledge Shark Wave has a bundle format defined in the companion document.
These are bundled into a tar file that can be dynamically retrieved into the
users Semantic Marker® app.
With the SMART Matrix, a new graphical password can be created by making the syntax such as it passes on a URL easily. For example:
@matrixPassword(1.1.QHmwUurxC3.1728862503265,1.2.QHmwUurxC3.1728862975699,1.3.QHmwUurxC3.1728863096071,2.1.QHmwUurxC3.6240511737410)
A new feature to get the Agents really going, is the addition of the trigger. This is a condition that when met lets
the Agent run it's KSApplet (Knowledge Shark Applet) - a set of parameterized IoT messages. These are extracted from the Agent's futureSM field. The syntax supported
at present (and when in the DOCFOLLOW mode) are a follows:
@trigger( <trigger_set> <condition> <trigger_val)
where
condition is ==, > , < & and a couple still in work
In addition, a hard coded device name can be used (although it reduces the generic nature. It would make sense for things like light#5 that all houses might have.)
@device(<deviceName>)
@device($scannedDevice)
The $scannedDevice or others with the $ will be meta-names. This means they have local meaning. Seeing the scannedDevice message is an example,
so this will re-evaluate the device name dynamically to whatever the scannedDevice is.
These agent scripts will be added to the event loop so when set val messages arrive, this list of trigger objects will be evaluated. If they match the condition, their parameters are substitued dynamically and run (as http IoT messages). This way
these KSApplets can run for the all users but instantiated for the current user and password.
(first run 11.13.24 watching the M5AtomSocket light switch on and off (toggle) when a condition of disk == 94 arrived.)
See SMART Agent