Platform Supported
Language Stats
Repo Stats
Code Stats
Deployments
Activity
Development and Maintenance
Reach Out
Install
python -m pip install jarvis-ironman
Initiate
import jarvis
if __name__ == '__main__':
jarvis.start()
-
MacOS
Tested on macOS High Sierra, Mojave, Catalina, Big Sur, Monterey and Ventura*System Preferences
→Security & Privacy
→Privacy
- Click
+
sign and add the preferredIDE
andTerminal
in the following sections in left pane.Microphone
- Required to listen and respond.Accessibility
- Required to use key combinations for brightness and volume controls.Camera
- [Optional] Required only during face recognition/detection.Automation
- Required to controlSystem Events
and other apps like Outlook and Calendar.Files and Folders
[OR]Full Disk Access
- Required for allFileIO
operations.
-
Linux
Tested on Ubuntu 22.04 LTS- Store the host machine's password as the env var:
ROOT_PASSWORD
- Unlike macOS and Windows,
Ubuntu
does not have app specific permissions.
- Store the host machine's password as the env var:
-
Windows
Tested on Windows 10Settings
→Privacy
Microphone
- Required to listen and respond.Camera
- [Optional] Required only during face recognition/detection.- Unlike
macOS
,Windows
pops a confirmation window to Allow or Deny access to files and folders.
- Install Anaconda or Miniconda, VisualStudio C++ BuildTools, and Git
- Make sure C++ build tools are installed completely and restart
- Add anaconda/miniconda scripts location to
PATH
in Environment Variables
Camera
from jarvis.modules.camera import camera
if __name__ == '__main__':
cam_object = camera.Camera()
print(cam_object.get_index()) # Get index id of each camera
print(cam_object.list_cameras()) # List all connected cameras
Text to Speech
from jarvis.modules.speaker import speak
if __name__ == '__main__':
speak_object = speak.Speaker()
speak_object.run()
print(list(speak_object.get_all_voices()))
Speech to Text
import asyncio
from jarvis.modules.microphone import recognizer
if __name__ == '__main__':
asyncio.run(recognizer.main())
Environment variables are loaded from a .env
file and validated using pydantic
More on Environment variables
- ROOT_PASSWORD - System password to get the system vitals and run other
sudo
commands. (mandatory forLinux
) - NAME - Name which Jarvis should address the user by. Defaults to
Vignesh
- TITLE - Title which Jarvis should address the user by. Defaults to
sir
- LIMITED - Boolean flag to run only the main version of
Jarvis
skipping background processes. Defaults toFalse
(enforced based on the number of CPU cores) - PLOT_MIC - Boolean value to show microphone usage in realtime. Defaults to
True
-
WAKE_WORDS - List of wake words to initiate Jarvis. Defaults to
['jarvis']
(Defaults to['alexa']
in legacy macOS)
⚠️ Jarvis has limitations on the wake words as it relies on ML libraries for wake word detection. -
VOICE_NAME - Name of the voice supported by the OperatingSystem. Defaults to the author's favorite.
-
SPEECH_RATE - Speed/rate at which the text should be spoken. Defaults to the value from
pyttsx3
module. Typically200
To add more voices
macOS:
- System Preferences → Accessibility → Spoken Content → System voice → Manage Voices...
Windows:
- Settings → Time & Language → Speech → Manage voices → Add voices
-
SENSITIVITY - Hot word detection sensitivity. Allowed range: [0-1] Defaults to
0.5
-
LISTENER_TIMEOUT - Timeout in seconds until which the listener should wait for speech. Defaults to
3
-
LISTENER_PHRASE_LIMIT - Timeout in seconds until which the listener will remain active. Defaults to
None
-
RECOGNIZER_SETTINGS - A JSON object with customized speech recognition settings.
Custom settings for speech recognition
These are customized according to the author's voice pitch. Please use recognizer.py to figure out the suitable values on a trial and error basis.
Sample settings (formatted as JSON object)
RECOGNIZER_SETTINGS
:'{"energy_threshold": 1100, "dynamic_energy_threshold": false, "pause_threshold": 2, "phrase_threshold": 0.1, "non_speaking_duration": 2}'
Description
energy_threshold
: Minimum audio energy to consider for recording. Greater the value, louder the voice should be.dynamic_energy_threshold
: Change considerable audio energy threshold dynamically.pause_threshold
: Seconds of non-speaking audio before a phrase is considered complete.phrase_threshold
: Minimum seconds of speaking audio before it can be considered a phrase - values below this are ignored. This helps to filter out clicks and pops.non_speaking_duration
: Seconds of non-speaking audio to keep on both sides of the recording.
- CAMERA_INDEX - Camera index that has to be used. Run camera.py to get the index value of each camera.
- SPEAKER_INDEX - Speaker index that has to be used. Run peripherals.py to get the index value of each speaker.
- MICROPHONE_INDEX - Microphone index that has to be used. Run peripherals.py to get the index value of each microphone.
- DEBUG - Boolean flag to enable debug level for logging. Defaults to
False
- LOG_RETENTION - Number of days to store the logs. Defaults to
10
- DISTANCE_UNIT - Unit in which speed/distance should be measured. Defaults to
miles
- TEMPERATURE_UNIT - Unit in which temperature should be measured. Defaults to
fahrenheit
- GIT_USER - GitHub Username
- GIT_PASS - GitHub Token
- BIRTHDAY - Birth date in the format DD-MM - Example:
24-April
- WEATHER_API - API Key from openweathermap
- NEWS_API - API Key from newsapi
- MAPS_API - API Key for maps from Google
- OPENAI_API - API Key from openai
- OPENAI_REUSE_THRESHOLD - This is the threshold for sequence matching repeated requests. Defaults to
None
Setting the
OPENAI_REUSE_THRESHOLD
(range:0.5
to0.9
) will try to avoid hitting OpenAI API more frequently for repeated identical requests.
This threshold is theratio
of acceptable result forSequenceMatching
between a new request and historically identical request.
- GMAIL_USER - Gmail account username to send and read emails.
- GMAIL_PASS - Gmail account password to send and read emails.
- OPEN_GMAIL_USER - Alternate gmail account username to send an SMS. (Defaults to
GMAIL_USER
) - OPEN_GMAIL_PASS - Alternate gmail account password to send an SMS. (Defaults to
GMAIL_PASS
) - RECIPIENT - Email address to which the emails from jarvis have to be received.
iOS integrations
- ICLOUD_USER - iCloud account username/email.
- ICLOUD_PASS - iCloud account password.
- ICLOUD_RECOVERY - Recovery phone number to activate lost mode on a target device - Example:
+11234567890
- PHONE_NUMBER - To send SMS from Jarvis - Example:
+11234567890
Calendar/Meeting integrations
- ICS_URL - Shared calendar URL to get meetings information from. Should end with
.ics
- MUTE_FOR_MEETINGS - Boolean value to disable the listener during meetings. Defaults to
False
- EVENT_APP - To read events from
outlook
orcalendar
application inmacOS
. Defaults tocalendar
💡 Whencalender
is used, the name of the calendar within theCalendar.app
should be Jarvis
Background scans [Defaults to 1 hour]
- SYNC_MEETINGS - Interval in seconds to generate
meetings
information using anics
URL. - SYNC_EVENTS - Interval in seconds to generate
events
information usingcalendar
oroutlook
application.
Scheduled
-
WEATHER_ALERT - Time (in 12h
%I:%M %p
format) when the weather alert should be fired - Example:09:00 AM
- WEATHER_ALERT_MIN - Degrees below which alert has to be fired.
- WEATHER_ALERT_MAX - Degrees above which alert has to be fired.
Alerts in SMS and Email if temperature is higher than
WEATHER_ALERT_MAX
or lower thanWEATHER_ALERT_MIN
or severe weather warnings.
This feature can also be enabled fromautomation.yaml
by using the keywordweather
in phrase for thetask
-
CRONTAB - Runs external tasks using cron expressions. Needs to be stored as env var.
Sample value
[ "0 0 * * 1-5/2 find /var/log -delete", "0 5 * * 1 tar -zcf /var/backups/home.tgz /home/" ]
- WIFI_SSID - SSID of the wireless connection.
- WIFI_PASSWORD - Password for the wireless connection.
- CONNECTION_RETRY - Frequency in seconds to check for an active internet connection. Defaults to 10 seconds.
VPNServer integration
- VPN_USERNAME - Username to create vpn-server. Defaults to profile username.
- VPN_PASSWORD - Password to authenticate vpn-server. Defaults to profile password.
- VPN_DOMAIN - Domain name for the hosted zone.
- VPN_RECORD_NAME - Alias record name to access VPN server.
Car Controls - Applies only for JLR vehicles subscribed to InControl
application.
- CAR_EMAIL - Email address to log in to InControl API.
- CAR_PASS - Password to authenticate InControl API.
- CAR_PIN - InControl PIN.
Garage Controls - Applies only for garages using MyQ garage controller.
- MYQ_USERNAME - Email address to log in to MyQ API.
- MYQ_PASSWORD - Password to authenticate MyQ API.
Telegram Bot integration
- BOT_TOKEN - Telegram BOT token.
- BOT_CHAT_IDS - List of userID/chatID to authorize.
- BOT_USERS - List of usernames to authorize.
-
SPEECH_SYNTHESIS_TIMEOUT - Timeout to connect to the docker container that processes text to speech requests.
-
SPEECH_SYNTHESIS_VOICE - Voice for the speech synthesis model. Defaults to author's favorite.
-
SPEECH_SYNTHESIS_QUALITY - Quality of speech synthesis conversion. Defaults to
medium
.To enable independent speech-synthesis
docker run \ -it \ -p 5002:5002 \ -e "HOME=${HOME}" \ -v "$HOME:${HOME}" \ -v /usr/share/ca-certificates:/usr/share/ca-certificates \ -v /etc/ssl/certs:/etc/ssl/certs \ -w "${PWD}" \ --user "$(id -u):$(id -g)" \ thevickypedia/speech-synthesis
💡 Speech Synthesis can run on a docker container for better voices but, response might be negligibly slower. If you don't have docker installed or simply don't want to use it, set the
SPEECH_SYNTHESIS_TIMEOUT
env var to 0. This is also done automatically if failed to launch a docker container upon startup.
- OFFLINE_PORT - Port number to initiate offline communicator. Defaults to
4483
- OFFLINE_PASS - Secure phrase to authenticate offline requests. Defaults to
OfflineComm
- WORKERS - Number of uvicorn workers (processes) to spin up. Defaults to
1
Stock Portfolio
- ROBINHOOD_USER - Robinhood account username.
- ROBINHOOD_PASS - Robinhood account password.
- ROBINHOOD_QR - Robinhood login QR code
API Features
- ROBINHOOD_ENDPOINT_AUTH - Authentication token to access the robinhood portfolio which is generated every hour.
- SURVEILLANCE_ENDPOINT_AUTH - Token to access webcam live feed via Jarvis API.
- SURVEILLANCE_SESSION_TIMEOUT - Session time out for
/surveillance
. Defaults to 300 seconds. - STOCK_MONITOR_API - API key(s) to skip OTP validation for
/stock-monitor
endpoint.
Entry should be a dictionary of with email as
key
andtoken
as value. Eg:'{"[email protected]": "@lfaROm$o"}'
Custom Feature Flag
- AUTHOR_MODE - Enables reverse proxy tunnel (to expose API endpoint), robinhood portfolio scan (every hour) and stock monitoring (every 15 minutes). Defaults to
False
Jarvis can send on demand notifications using a contacts.yaml
file stored in fileio
directory. Uses gmail-connector for SMS and email notifications.
Setup Instructions
Note: Jarvis currently supports sending emails only when the
contacts.yaml
file is present, however phone numbers can be used directly.
phone:
Tony: 0123456789
Thor: 1234567890
email:
Eddard: [email protected]
Aegon: [email protected]
Jarvis supports MagicHome
for lights, LGWebOS
and Roku
for TVs.
This feature requires a
smart_devices.yaml
file which should be stored within thefileio
directory.
-
TV hostnames should include the brand name [
LG
/Roku
] to distinguish the modules accordingly.- This will be set by default, if yours doesn't include the brand name change it in the TV settings.
-
To wake up
Roku
TVs using MAC address, make sure theBandwidth saver
feature is turned off under,Settings/Network/Bandwidth saver >> Off
-
For first time users on
LGWebOS
TVs, there will be a prompt on the TV to accept the connection request.- Once the connection request is accepted a client key will be generated and logged.
- Please make sure to store this in
smart_devices.yaml
file to avoid repeated connection prompt.
Setup Instructions
- The source file should have an initial key to distinguish between
Lights
andTV
- The name used in the keys (for both lights and tv) will be the identifier when an action is requested.
- Lights should be a dictionary of identifier and a list of hostnames.
- TVs should be a nested dictionary of multiple parameters.
- The source file (
smart_devices.yaml
) should be as following:
Lights:
bedroom:
- 'HOSTNAMES'
hallway:
- 'HOSTNAMES'
hallway basement:
- 'HOSTNAMES'
kitchen:
- 'HOSTNAMES'
living room:
- 'HOSTNAMES'
TV:
living room tv:
hostname: 'LGWebOSTV'
client_key: 'CLIENT_KEY' # Generated automatically if unavailable
mac_address: # Mac addresses can either be a string or a list
- 'WIRED_MAC_ADDRESS'
- 'WIRELESS_MAC_ADDRESS'
bedroom tv:
hostname: 'RokuTV'
mac_address: 'MAC_ADDRESS'
Jarvis can execute offline compatible tasks at pre-defined times without any user interaction.
This feature requires an
automation.yaml
file which should be stored within thefileio
directory.
Setup Instructions
The YAML file should be a dictionary within a dictionary that looks like the below.
OPTIONAL: The key, day
can be a list
of days, or a str
of a specific day or simply a str
saying weekday
or
weekend
when the particular automation should be executed.
Not having the key
day
will run the automation daily. Date format should match exactly as described below.
06:00 AM:
- task: set my bedroom lights to 50%
day: weekday # Runs only between Monday and Friday
06:30 AM:
- task: set my bedroom lights to 100%
day: # Runs only on Monday, Wednesday and Friday
- Monday
- wednesday
- FRIDAY
08:00 AM: # Runs only on Saturday and Sunday
- task: set my bedroom lights to 100%
day: weekend
09:00 PM: # Runs daily (can take both list of dict and dict as argument)
task: set my bedroom lights to 5%
12:00 AM: # Even performs tasks that are not supported via voice commands
- task: restart all background processes
- task: turn off all lights
day: weekday
Jarvis supports both internal and external background tasks to be scheduled.
- Jarvis can run internal tasks at certain intervals.
This feature requires a
background_tasks.yaml
file which should be stored within thefileio
directory.
Setup Instructions
- seconds: 1_800
task: just turn off all lights # Runs every 30 minutes - 'just' flag retains the lights' last setting (eg: brightness or color)
ignore_hours: # Ignore the schedule at 5 AM and 10 PM
- 5
- 22
- seconds: 10_800
task: remind me to drink water # Runs every 3 hours ignoring the hours specified
ignore_hours: "21-6" # Ignore the schedule between 9 PM and 6 AM
Jarvis can execute tasks as a simulation to test the required functions and send an email with the results.
This feature requires a
simulation.yaml
file which should be stored within thefileio
directory.
Setup Instructions
The YAML file should be a list of phrases within a dictionary that looks like the below.
meeting_event:
- get me the events from my calendar
- what meetings do I have today
Jarvis can execute function(s) directly based on a custom condition map, via both voice and offline communicators.
- The condition mapping may contain one or more functions to be executed for a single keyword/phrase.
- These functions are executed as ordered in the mapping file and responses will be delivered as each task is done.
- For offline communicators, in case of multi-function mapping, the responses are gathered and then delivered at once.
This feature requires a
conditions.yaml
file which should be stored within thefileio
directory.
Setup Instructions
The YAML file should be a dictionary within a dictionary that looks like the below.
lumos: # custom keyword
lights: turn on all lights # function_name: phrase passed as argument
knox:
lights: turn off all lights
television: turn off bedroom tv
fire up the chopper: # custom phrase
garage: open garage
car: start car
UserWarning
if invalid function name or phrase is entered in the mapping file.
Please refer to the wiki page for API usage, access controls, env variables, features' overview and demo videos.
Please refer to the FAQs section of the wiki.
Docstring format: Google
Styling conventions: PEP 8
Clean code with pre-commit hooks: flake8
and
isort
Requirement
python -m pip install changelog-generator
Usage
changelog reverse -f release_notes.rst -t 'Release Notes'
PreCommit
will ensure linting, and the doc creation are run on every commit.
Requirement
pip install sphinx==5.1.1 pre-commit recommonmark
Usage
pre-commit run --all-files
https://pypi.org/project/jarvis-ironman/
https://thevickypedia.github.io/Jarvis/
© Vignesh Sivanandha Rao
Licensed under the MIT License