Pleiades Robotics is setting up a workspace at the new COVE development on Halifax Harbor. COVE (Center for Ocean Ventures and Entrepreneurship, link: https://coveocean.com/) is located in a former Coast Guard station. Dozens of organizations are involved. Pleiades will be developing applications for our Spiri drones to monitor ocean health, assist in marine navigation, and support surface and underseas missions from the air. This continues with work we have been involved in at Dalhousie University.
In a break from our usual posts about manufacturing and technical progress, we want to share these pictures of Spiri on a unique mission.
Last week, the European Space Agency welcomed a team from the MIST lab at the Polytechnique de Montréal to the island of Lanzarote in the Canaries. With them came a flock of Spiris that the MIST team, with Pleiades support, have been training to organize into autonomous formations. The drones, once airborne, require no human control to arrange themselves, and no ground- or satellite-based infrastructure to communicate. The successful mission is the culmination of more than three years of collaboration between the MIST team, led by Prof. Giovanni Beltrame, and Pleiades.
The European Space Agency (ESA) has set its sights on the moon and Mars. Images are courtesy of the ESA. For more information, check out these links:
We wish our American friends and supporters a very happy Thanksgiving. We have a great deal to be thankful for, and above all the support of everyone who has taken a stake in our company and stood by us. To Mars and beyond!
We're going to show you some of the work we've been doing on Spiri's machine vision, but first here is a quick update on production and shipping.
The last batch of robots to ship out seem to have stood up well to use in the university settings where most of them went. In the next six weeks we expect to be running another batch through assembly, and we will work down the list of backers, letting you know when yours is ready and verifying your current address.
There are a few small changes in the next batch: we are switching the ESC and motors to newer versions.
We have been working on Spiri's installer and on its vision system.
The installer is almost where we would like it to be. Our ongoing work is to remove the Linux desktop GUI, reduce the ROS packages to those relevant to Spiri, ensure everything that uses Python has Python 3 bindings, and provide efficient vision pipelines that shift more of the burden onto the GPU. We also want to make sure we provide you with good examples, in C++, in shell scripts, in ROS, and in Python of basic functions, such as running the cameras and moving the robot.
We have been working on configuring a system for simultaneous location and mapping (SLAM) on board. It allows Spiri to navigate using its cameras, and this can be helpful where GPS is unavailable, such as indoors, and it will eventually fuse with GPS measurements to give you much more precision in navigation. Here is a screenshot of Spiri identifying "features" (marked in green) it will use to track its position in the room.
We have some tuning to do, but fundamentally, it is working.
You can also record video and images with the cameras, and stream them back over Wi-Fi to your laptop or other device in real time. We have image rectification working in real time, and we are working on color correction. Here is a rectified, color corrected image from a Spiri video stream.
We are adding more quality control to our assembly process, and will be doing another batch of robots this October once parts come in.
We are also working on an installer set, both for the robot itself, and also for a development laptop and a simulator. This means you will be able to start programming for Spiri before you get it, and try your programs out before flying.
For today, I want to share a video. What you're seeing is part of the work ongoing at McGill University, where they have been working with the release version of Spiri over the summer. Their work is on maneuvering, collision recovery, and being able to fly or recover to fly in extreme situations.
In this clip, Spiri flips over, and then hovers calmly up while upside-down. Symmetrical propellers are used, and the ESC firmware is altered to allow both directions of spin. Once ready for release, we'll show you how to do this, too.
We had a manufacturing related problem with the last batch of robots we were preparing. The wrong kind of bolt was used on a number of motors, causing them damage. We got replacements and now we are back to work on these Spiris.
We aren't just putting Spiris together with this batch, but also proving unit and assembly tests. We intend, after a couple iterations in batches of 8-12, we will have a systematic line, and every part of every Spiri going out will have been tested and verified.
The issue on the line delayed our shipping. Once this batch is finished, and after some tests of the robots' ability to work as a group, we will resume shipping to you.
Besides the work on our assembly processes, we are also working on the installer, camera improvements, default settings, a disk image, and soon will be cleaning up and organizing our GitHub repos to make it easier for you to work with Spiri once you have it.
Many of you may be at different addresses from what we have on file, so prior to shipping we'll be in touch to get your location.
A quick update to let you know that the next batch of Spiris are going to ship around June 15. Some improvements to the robot include:
- elastomer holders for the cameras
- better sealing along the shell
- improvements to the camera driver and control
- improvements to Wi-Fi reliability
- improvements to the wiring
- tuned flight profiles for indoor and outdoor flight, manned or autonomous
We will be in touch with the next Kickstarter backers in line to confirm your shipping address. A big thank you to all our backers for your support and patience over the years!
We've had some feedback from the group of university labs we sent the last small batch to. Aside from the wiring issue mentioned in the last update, there are a few further mechanical and wiring corrections we decided to make. These should make your robots more reliable, and less likely to break difficult parts to repair. The next batch will be sent half to Kickstarter backers, half to commercial and academic partners.
There were a couple of crashes in the lab. Something we recommend, is if you have a crash with impact on the battery container, to treat those batteries with a lot of suspicion.
We're working in the meantime on improvements to the vision system and low level kernel task management. We're also tweaking flight parameters, partly through trial and error, both for outdoor and indoor (GPS denied) flight.
Here is a link to some raw footage of a flight test of the Acro mode. Our lead mechanical designer is flying the Spiri in this video by remote control, and demonstrates in-air flips. Just a fun video to show that Spiri is quite agile, on top of everything else.
We built and shipped a small batch of Spiris over the last few weeks, but we haven’t got as far into production as we hoped. Technically, we are ready to go. We have been waiting for a tranche of funding from our government and private partners to launch the beta at full scale. In the meantime, we are doing our best to bootstrap.
Along those lines, we are preparing a production test of 20 Spiris, which will go out to Kickstarter backers and research partners. Our aim is to get to a production volume of one Spiri per day built and shipped, and to scale up from there after we bank our coming round of funding.
We have only recently overcome the last hurdle with our camera driver, which has plagued us for months and months. In the image below, you see on screen a live feed from one of Spiri’s cameras (which have been pulled out of Spiri for upgrade work and testing). We have also been working on bug fixes, feature enhancements, and documentation. Here are some highlights:
- Cameras are now enabled for gstreamer and (for application development) libArgus, which means access to GPU optimizations on the TX1 and TX2;
- Cameras, by the way, connect directly via CSI rather than USB, which is much better for bandwidth, and frees up your USB port for anything else;
- Two ways to do autonomous flights: either use “mission” mode on the flight controller, where you upload a pre-written mission (e.g. through QGroundControl), or use “offboard” mode, where you send navigational commands directly from the TX1 or TX2 to the flight controller.
We have created a settings list for the flight controller that works well on Spiri. We also have the ORB_SLAM2 algorithm running, which Spiri can use to map out the features in its surroundings and locate itself within them.
We have run out of technical and manufacturing problems, and we are now sorting out the launch funding. Spiri is ready.
On a flight test about a week ago, I (Patrick) flew Spiri into a telephone pole. It was human error: I thought Spiri was getting too close to the pole for comfort, so I took the controls, and promptly flew it in the opposite direction I intended, straight into the pole. It hit hard, bounced off, and stayed in the air. The top of the shell took some damage, but I was impressed at how well the flight controller reacted, everything from the motors activating their safety stop and restart, to the recovery to stable hover. But don’t tempt fate or do this on purpose!
We have been working towards pilot projects in agriculture, local first responders, construction, and utilities. Right after the telephone pole accident, in fact, we performed a successful demonstration for police, fire and medical responders in the town of Vineland, NJ. Spiri, this time without errant human interference, carried out three beautiful pre-programmed missions. We are seeing more and more interest in using robots for these kinds of applications, and more and more acceptance, at least conceptually, of autonomous flight and easier regulations. It is going to be an exciting time for robotics!
We have been locking down the installed firmware and software, and working on the assembly process since the last update. I’m happy to say, we got the camera system working, and in the intervening time, there have been great updates to the underlying operating system kernel. Some of the improvements will help machine vision by strengthening the GPU optimization on Spiri. One very immediate improvement is that the same installer now works for both the TX1 and TX2, so if any of you want to upgrade to a TX2 Spiri, you can do it without compromise or worrying about backwards compatibility.
At the assembly facility today, I took a picture or Spiri in different stages: loose parts, propulsion system, flight control system, main computer, and then fully built. It has been a long journey and you have all waited a long time to see an assembly line of your Spiris.
There are still some glitches to work out, but some of you will be getting a robot in the next weeks, and we will be here to help with bugs, configuration issues, and also to give tips and learn about what you are doing.
In this update, I'm going to talk about the software installed on Spiri and about the different user interfaces available to you.
Flight Control Firmware
4-in-1 ESC – the Littlebee board that governs speed control for the four motor drivers runs BHeliS with d-shot software. This new digital controller increases the number of motor speed adjustments per second, making for smoother flight.
Flight Controller – the PixRacer module runs custom PX4-Spiri firmware specially modified for Spiri to enable the fusion of visual data and emergency recovery processes, among other changes, into the core flight control system.
Main Computer System
TX1 (or TX2 upgrade) by Nvidia incorporates a quad-core ARM CPU with a 256 CUDA Maxwell (or Pascal) GPU, excellent memory, bus, and core features.
Ubuntu 16.04 with Spiri-L4T-1.0 kernel – this is the basic OS and enabler for essential hardware drivers, including our stereo global shutter cameras.
ROS Kinetic – this is a high level method for broadcasting and receiving information among robots or processes within a robot. There are numerous open source ROS projects which will work on Spiri.
Robot Web Tools with Robot Management System – this powers the web page interface with Spiri
MAVROS, MAVProxy – this governs the exchange of data between processes on the main computer and on the flight controller, for example, the communication of SLAM (simultaneous location and mapping) data to the flight controller where it is merged with GPS and other data to increase the accuracy of Spiri's position estimates.
ORB_SLAM2 with Pangolin – this enables SLAM on Spiri
Various tools and dependencies, including gstreamer, Git, Pangolin, Eigen3, and certain deep learning tools, are also on Spiri.
There will be occasional software and firmware updates. The update process depends on which part of the robot is affected.
You may, once every three months or so, receive an update to program into the flight control board. For this you will need to remove the top shell of Spiri and connect a USB cable between your PC or laptop and the flight control board on the robot. A free, open source ground control program, such as QGroundControl, which will guide you through the process and we will provide support documentation.
You will not need to flash the ESC firmware in any circumstance we foresee, but it is possible to do so. We will provide support documentation.
Updates to any of the software on the main computer (Linux on the TX1) will be sent to you for a basic yes/no confirmation to install. Your files will all be stored on the removable SD card inserted into the carrier board inside the robot. The operating system and all updates to it are sequestered on non-removable storage on the TX1. This should ensure ongoing compatibility.
Filesystem, Removable Storage, On- and Off-Board Storage
8Gb SD card in Flight Control Board (removable) – this is Spiri’s “black box”, the log of all sensor data collected during each flight
16Gb SD card in Main Computer Carrier (removable) – this is where your home/user directory is, it is your on-board app and media center
16Gb solid state drive on Main Computer – this is where the operating system and all factory-installed software is, along with a home/spiri directory for non user-specific access
spiri.net/user space – coming soon, basic and premium online media, app, and backup storage
RC – any receiver compatible with the PX4 software and a transmitter will work with Spiri, and we provide instructions for one example, the Orange T-Six and R110XL receiver. RC will override any autonomous controls and put you alone in control of Spiri’s flight
Ground Station – any ground control software on your PC or laptop (we provide support for QGroundControl) will give you basic location, direction, and map interfaces
RMS – Robot Management System permits video streaming and multistage mission planning from your PC, laptop, tablet or smartphone over Wi-Fi to Spiri, which serves a webpage to your browser of choice. The browser is the user interface for RMS to make and send direct commands, flight plans, and broad autonomous missions while viewing video and other data in real time
ssh – secure shell from the unix command line gives you a direct line to Spiri to give it a mission, run a script, app, or other software, and can be used concurrently with RC, RMS, or other modes of communication and control. This is also one way to add to or modify Spiri's programming.
In addition to these user interfaces, we have programmed Spiri to interact with you and its environment in other tangible ways.
Spiri can see AR codes, and we have programmed a set of “runes” it reacts to
Spiri can respond to voice commands via smartphone
Spiri can respond to MIDI messages, that is, you can encode commands in musical figures
We expect these features to be in pre-release review when we begin shipping, and we will announce when they are ready for general release.
We maintain a GitHub repository where mirrors of all the critical systems on board can be found, along with our own software. A public app center is in the works.
Our support pages will contain detailed information on communications and user interfaces, and on how to build apps and features into Spiri if you are a programmer.
We are correcting a few driver and boot sequence issues on Spiri. Next week I'll have more information for you.
part of RMS menu
In this update, I'm going to talk about the work we've been doing over the past few months, and the choices we’ve made in order to begin shipping. Three points I want to make before getting down to details are:
Anyone who supported us at the Spiri or more level will get one of the first 1,000 Spiris we make;
No matter what tier of support you provided, Spiri is a better robot than you can buy today for the same amount;
We will be available for end user support and we will continue to develop features and patches for download. We have also designed Spiri with repairability and ease of hardware upgrade in mind.
Today, I want to tell you about (1) our plan for shipping Spiri, and (2) what will be in your Spiri. I also want to let you know that from here on, we will post weekly updates on specific topics, howtos, and retrospectives on the project in general. We will be ramping up our communications, and eventually transitioning to a blog on our website.
We are going to do a staged release of Spiri, starting with a dozen close research partners, original equipment manufacturers and high tier Kickstarter backers. This will help us populate our support pages and catch bugs in the software, vision systems, communications, control, or web tools before hundreds of people experience them. It will also give these researchers and us a chance to port work from other projects or from previous versions of Spiri onto this release version, so that everyone else will have those features, too.
Once that phase is finished, we intend to accept and fulfil new orders while simultaneously fulfilling Kickstarter rewards and pre-orders. We intend to maintain roughly a 1-to-1 ratio of new orders to pre-orders and rewards. This is the best compromise I can make between maintaining your priority in line and satisfying the conditions of the company’s financial backers.
We went through two prior iterations of Spiri during this project. The first had too little computational power and the second had too much bulk. We believe the release version is just right. The primary equipment we selected is as follows:
PixRacer flight controller with additional GPS, compass, and Terabee rangefinder
Nvidia TX1 (or TX2 upgrade) mounted on a ConnectTech carrier board
Two Leopard Imaging M021C cameras with stereo vision enabled
Since the project began, we have made many improvements to Spiri but we have also traded off some of the features we originally wanted. I want to compare and contrast the original version of Spiri we announced on Kickstarter to the release version.
Flight time has gone from 10 minutes to 22 minutes
Typical GPS satellite locks outdoors has gone from 8-12 to 12-16
Rangefinder now points down instead of forward
Rangefinder limit has gone from 2m to 5m
Stereo cameras now global shutter, improved low light, double the throughput
Flight control sensors doubled
Switched from basic filters to PX4-based controller
Added MAVLink support in ROS
Doubled number and strength of Wi-Fi antennas
Upgraded USB output to USB3
Added I2C ports, PCIE slot
Factor of four improvement in CPU, memory, and storage
Factor of eight improvement in GPU
About 500g heavier (from about 1,000g to about 1,500g)
Excluded the propeller guards
Excluded the wireless autonomous recharging perch
Target retail price has increased significantly from $600 to $2,500
We plan to revisit the propeller guards and wireless recharging. For now, as we see, it, you've waited long enough. The retail price is higher than any of the Kickstarter tiers where a Spiri was the primary reward.
I know some of you are wondering why we’ve been so quiet for the past few months. I will write a more detailed project retrospective in 2-3 weeks. Briefly, our release was stalled because of an issue with the camera driver boards, and because of a delay in the government contribution to our funding. Week by week, I didn’t feel like I had signficant new news to share. We were making progress in other ways, finalizing/upgrading parts, working the assembly process, but I couldn’t make a shipping announcement until the funding and hardware issues were settled, which always seemed just around the corner. I will get back to this in the project retrospective in a couple of weeks, I just wanted to acknowledge it today.
In next week’s update, I am going to talk about the software on board Spiri, and the many different user interfaces that are available to you.
Thank you again for your support and patience.
the evolution of Spiri