Software for sensor devices.
- Measure uniformity of growth.
- Provide curve of growth in time.
- Algorithm is plant-specific.
- Plant in tray is known.
- Measurements across different plant species are not comparable.
- Day of growth is known. Starting day and hour must be known.
- White, constant light.
- Substrate is white or black and plastic.
- Distance and angle are fixed.
- Definition of architectural aspects (image storage, JSON, etc.)
- Design enclosing case
- Hardware specification document
- Design and implement controls on web interface
- Algorithms: segmentation, analysis, calibration
-
Commented source code -
Operation guide
Keep record of these (info.sh):
- SD serial
- SD cid
- Pi serial
- eth0 MAC address
- usb0 MAC address
- wlan0 MAC address
DHCP on usb0, eth0 and wlan0.
Change via script:
- Hostname / hosts file (change_hostname.sh)
- Wifi SSID and password (wifi.py)
- List all pictures on S3
- Check if local file exists
- Download pictures that are not here yet
- Group images with the same hostname and batch identifier
- Open downloaded images
- Cython segmentation function
- Optionally download images from S3
- Display multiple images with the discarded areas, for debugging purposes
- Stable biomass bounding box (also useful for cropping)
- Unit tests, integration tests
- Modularize the image processing pipeline
- Test median blur and blur
- Segmentation: model hole color and dead leaves color
- Segmentation: model substrate color
- Holes: consider their hierarchy
- Segmentation: multi-dimensional k-means clustering as a tool
- Lab, Luv, YUV color spaces
- Superpixels
- Probability map instead of binary mask: likelihood of a pixel to belong to a class based on color and spatial location (neighborhood); blur to diffuse probability
- Find biomass dominant tone based on saturation
- Detect if scene is dark
- Hires image and charts
- Find biomass segmentation algorithm params automatically
- Upload MQTT
- Detect if scene is not static (motion detection)
- Substrate-based uniformity algorithm
- Store custom group name
- DrawSmoothChart() and DrawChart() do not draw phantom point at time zero
- Series --> batch
- MQTT only for group zero
- Limit spline propagation experimented in branch spline-parts-and-spline-until-break
- Hours of operation shown and set on web interface, per sensor
- Plant species and group as dip.py argument (in batch name)
- Use batch_species and legacy functions to select special algorithms
- Web chart: variable ticks, fixed 15 days period
- Download no longer downloads also .csv files
- Make upload of csv optional
- Web output of processing (local network and Internet)
- List and link all available sensors and batches (including previous) *.csv for one group
- Directories prior/, downloaded/, timelapse/, CSV/ and possibly other dirs need group string
- Create group subdir under downloaded/, prior/, CSV/, timelapse/
- Prior knowledge is stored
-
Preview to set focus and set view -
Launch dip.py on server with cron -
Link chart back to control panel -
A big chunk of zero.py (batch_start, minutes...) is general and should be taken out -
Send mail notifications -
Default, generic routine with just motion and brightness (to replace RoutineDisplay as default) -
Possible issue: Download from S3 error 'The read operation timed out' -
Possible issue: browser caches local csv file and does not update it even when refreshing -
Web chart: add URL parameter to highlight a specific moment -
Thumbnail size can be specified in URL (useful on local network) -
Comparison with average species curves -
Store all frames on sensor until end of batch (useful for processing on sensor) -
Create summary timeline with thumbnails -
Control panel and chart in same web page -
'Ping' via HTTP request -
Buttons -
Make console log available on the web -
Check valid hours; always on mode if scene is lit -
Image processing on sensor -
Timelapse with and without superimposed graphics, but only if there is at least one more frame processed -
Windowless cap and dip (windowed is a command line option) -
Hardware check based on MAC addresses and serials (see info.sh) -
Background subtraction -
Perspective warping for perspective invariant areas -
Servers list in git: an http request containing image name is done by cap.py until a server responds positively (web server in dip.py, or calling dip.py) -
Mask -
Textural information: spatial frequency function pixel by pixel, textons and their neighborhood histograms; Sobel, Scharr and Laplacian on these channels separately: saturation, brightness, luminance + Canny (blurred) -
Retain pixel probability from previous frames (running average) -
Segmentation: Otsu thresholding and adaptive thresholding -
Color analysis inside biomass: histograms -
Holes: pixel-by-pixel
- Set shutter speed
- Print all camera properties
- Save/Load camera properties
- Arrow keys select property and change its value
- Get/Set camera property value by name
- Set optimal camera properties programmatically
- Preview on/off
- Save image locally, timely
- Alternatively show a zoomed full resolution image (useful for focusing) or just a convenient thumbnail
- Add uptime to EXIF
- Add source code version to EXIF
- Print source code version when TAB is pressed
- Print date and time the process started (plus uptime) when TAB is pressed
- Add all camera properties to EXIF
- Determine best resolution for both camera versions
- Manage network failures (upload queue)
- Upload image to S3
- Decide whether to use same resolution for both models
- Detect camera hardware version
- Detect camera presence
- Unzoom when taking picture
- Disable preview automatically (when taking picture)
- Start cap.py at boot
- Update git repo, relaunch cap.py, reboot and shutdown remotely via script on the 4 sensors at once
- Integrate microphone
- Sensor color calibration == Find camera parameters automatically
- Web interface: number of images in queue
- Print 24 errors (with the average squared error)
- Web interface
- Click 4 times instead of 24, perspective grid
- Store date and time of crop start
-
Send sensor health warning (if something is wrong among cpu, memory, disk, temperature, camera properties, calibration...) -
Compare camera properties that are a result of color calibration and auto calibration across multiple sensors. It seems like shutter speed should be at least 5000. Probably find some custom target RGB values with new colorcheckers. -
Search brightness and contrast values (after the gains are good). Or make sure brightness is 50 and contrast is 10 and so on. -
Further test the exposure metering modes -
Possibly freeze awb but not iso, expo, shutter -
Camera on/off (250mA, no continuous mode)
# lsblk to list devices
# partition(s) must be unmounted (with umount; partitions, not the disk itself)
sudo dd if=/dev/mmcblk0 of=sdimage.img bs=4096 conv=notrunc # if=/dev/sdb bs=4M
# lsblk to list devices
# partition(s) must be unmounted (with umount; partitions, not the disk itself)
# be very careful with what follows, see https://www.raspberrypi.org/documentation/installation/installing-images/linux.md
sudo dd if=sdimage.img of=/dev/mmcblk0 bs=4096 conv=notrunc # of=/dev/sdb bs=4M
sudo kill -USR1 $(pgrep ^dd)
https://chrome.google.com/webstore/detail/vnc®-viewer-for-google-ch/iabmpiboiopbgfabjmgeedhcmjenhbla
sshfs pi@****.ddns.net:basil pi
fusermount -u pi
sshpass -p **** ssh pi@****.ddns.net 'bash -s' < do.sh
sshpass -p **** ssh [email protected].*** 'cd /home/pi/basil ; git pull ; sudo /sbin/shutdown now'
blueshift eth: 192.168.0.6 wifi: DHCP Pi NoIR Camera V2
redshift eth: 192.168.0.7 wifi: DHCP Pi NoIR Camera V2
noir eth: 192.168.0.8 wifi: DHCP Pi NoIR Camera V2
visible eth: 192.168.0.9 wifi: DHCP Pi Camera Module V2
Focus set at 350mm.
To look for local sensors:
sudo nmap -sP 192.168.1.0/24 | awk '/^Nmap/{ip=$NF}/B8:27:EB/{print ip}'
Resolution set to 1280x720 (16:9)
Append @/home/pi/basil/autostart.sh
to /home/pi/.config/lxsession/LXDE-pi/autostart
# 7 fps, other options include hd720, hd480
ffmpeg -r 7 -pattern_type glob -i '*.jpeg' -s hd1080 -vcodec libx264 timelapse.mp4
# with crop w:h:x:y
ffmpeg -r 7 -pattern_type glob -i '*.jpeg' -s hd1080 -vcodec libx264 -filter:v "crop=1050:871:857:776" timelapse.mp4
exiftool -keywords <filename>
Save username and password for 4 months:
git config --global credential.helper 'cache --timeout=10000000'
pip install Cython
sudo apt-get install python-scipy
pip install paho-mqtt
sudo apt-get install ffmpeg
pip install twisted
sudo pip install service_identity
pip install service_identity
The last command fails but then Twisted is happy anyway.
- Initial: Needs light. Must be done every time the sensor is turned on.
- Auto: Needs light + scene.
- Color: Needs light + colorchecker. Can provide final error(s).
Edit /etc/rc.local
and add the following lines above exit 0
:
# Disable HDMI
/usr/bin/tvservice -o
Other interesting settings in /boot/config.txt
.
git update-index --assume-unchanged FILE
Must be executed on each machine. That means this is part of the installation process.