13.77740
i love walking. whether through cities or over hills, walking around and seeing new things is one of my favorite activities.
i love creating. whether from fabric or wood or bits, the exploration of bringing new things into the world is no less delightful.
alas, the two activities are not so compatible.
it’s not that i need to do them simultaneously - though i certainly want to be able to get my thoughts down while walking a bit better. it’s that so much creation requires tools and materials that i simply can’t carry on my person.
the advent of maker spaces helps a lot actually - so long as i’m wandering around cities i can probably go to one for any woodworking, sewing, electronics, 3d printing needs. but the current state of the art in portable bitwrangling (which tends to include a lot of the design process) are laptops, tablets, smartphones.
the primary problem is ergonomics.
i don’t have debilitating repetitive strain injuries in my hands, yet, and i would like to keep it that way. i have had twinges using the laptop - and have since stopped using it as a laptop at all - and numbness after typing long messages on the smartphone. my back and shoulders also testify to my long use of laptops and a clipboard full of notes before that - though my posture when sewing is equally terrible. my current setup is a vast improvement over a laptop certainly… but it is hardly portable.
i have a plan though, and, as all the best plans are, it is modular.
The Display
my ideal display is something like a pair of fancy (if chunky) wrap-around sunglasses. i’d want minimal obstruction to my vision - i should be able to see as well as with normal sunglasses. but then the glasses can add additional light to my vision.
i suppose a monochrome display could be interesting to have built-in to clear lenses - blacking out pixels to make a display. though i am rather attached to having colours - and being able to use these in the dark.
in any case, the full range of view of each eye being 160°x175° and at 60 px/° to make it perfectly sharp… we’d need a 9600x10500 display - a bit higher resolution that is practical at the moment. but that whole range isn’t really necessary - most glasses don’t cover that! i mostly care about the area that i can see with both eyes - 120° wide - still 7200 pixels - fits in the width of an 8k display.
in reality, we have much smaller displays available - the various 1920x1080 OLED micro displays. in particular my current pick is the rokio max, that projects a 1920x1080 display onto a 50° diagonal, so a reasonable 44 px/°. of course the construction of it makes it look like it also blocks off a fair amount of your vision looking up - not exactly unobtrusive.
Other Outputs
audio to me. my current headphones need some significant rebuilding, but they provide an excellent audio experience. it’s the other half of the head-mounted great.
tactile. lots of experimentation to do here. there’s the standby of little vibrating motors spread across the body. seems like you could put a lot on a belt. maybe some on gloves.
there’s also the option of stimulating the touch neurons more directly. iirc it’s not the sensors themselves, but the nerve leading away from them are sensitive to induced currents. there are three types of sensors and one of them is sensitive to something like 400 kHz. i should find that paper again. this (possibly) has the advantage of being able to stimulate my fingertips without putting anything over them.
implanting magnets and putting wee electromagnets over them? probably a worse idea than electrostimulation. that at least wouldn’t muck with the magnetic positioning system.
video to others. for the most part i’d intend to keep a 15" 4k display in my backpack.
audio to others. there’s spare 3.5mm outputs, and usb ports. also the speakers on that display. i don’t feel any particular need for loudspeakers.
The Environmental Inputs
cameras. three of them. one in the middle and one just outside/above each eye. i have a binocular photography hobby waiting in the wings. the rokio max above doesn’t have any cameras - so there’s going to be at least a little hardware hacking going on. then like my normal external webcam for meetings?
microphones. ideally at least four with decent spatial distribution. location finding with signal delays and holographic recording! i think the rokio max has one. having one at each camera seems reasonable, and maybe three more on the headphones plus a dedicated talking mic?
ambient temperature, light levels, pressure, humidity, CO₂. i think i’d like these on the headphones? or maybe belt. not so much for the positioning as for the availability.
geopositioning system. GPS/Galileo/GLONSS/Beidou obviously on the headphones for good reception.
ambient ionizing radiation?
Position and Orientation Tracking
each piece of hardware should have an accelerometer/gyroscope/magnetometer chip. with the addition of three perpendicular magnetic field generators, each transmitting a different gold code, the magnetometer can provide additional absolute position updates to relieve the tendency to drift.
The User Inputs
well, there’s already the microphones. so if nothing else there’s voice commands / transcription.
at first there’ll surely be a split keyboard in the backpack.
eventually i’d like to make some very light gloves with an accelerometer/gyroscope/magnetometer chip on each finger, coupled with that magnetic absolute positioning system that could be really nice. those same sensors could go anywhere on the body for additional inputs.
gaze tracking would be nice. most of the smart glasses that do that are more integrated systems.
pen and touch inputs via the spare display in the backpack. i guess i could carry a mouse just in case?
Communications
terrestial radio. ideally something like a portable FlexRadio. practically speaking a hackrf or two. probably some dedicated higher power transmitters for HF-UHF. honestly these are probably backpack material.
wifi and bluetooth will be in whatever compute device. gb+ ethernet also. ideally i’d like an sfp+ port, but that looks to require spinning more hardware.
Compute
in the ideal world a 10 gram computer provides more compute than i know what to do with and consumes next to no power.
slightly more realistically, there is some sort of always-on low power computer that handles the various inputs and outputs and manages waking up the other computers and shuffling tasks to them as needed. it’d be cool if that computer could handle the basic user interface and pipe through interaction with a computer with a more powerful gpu. so basically a decent FPGA the lots of serdes, and either a good soft core or an attached hard core. then there are problems of boot or wake latency and scheduling and whatnot.
more practically still, just base everything on a minipc with a hefty processor and pay the power (and thus weight) budget. this has the advantage of actually being possible without spinning new hardware. current plan for this is to get a MinisForum UM790 Pro, which can be the bulk compute and storage for the previous plan someday.
In Summary
MVP (without making new hardware)
- Smart Glasses: Rokio Max (new)
- Mini PC: MinisForum UM790 Pro (new)
- Keyboard: Moonlander (current)
- Webcam: something cheap but decent (current)
- Battery??? (easy enough to put together pieces)
Bonus (without making new hardware)
- External Display (touch + pen input): INNOCN 15" 4k OLED (current) / HONGO 15" 4k OLED (new, vesa mount and no battery)
- Low Power/Receive Radio: HackRF (current)
- Environmental Sensor Platform: OSHW 2020 badge
- Soldering Iron: Pinecil v2 (current)
Areas for Improvement (making new hardware)
- magnetic positing tracking gloves (major input improvement!)
- FPGA+lightweight CPU primary human interface and sensor platform
Open Questions
Cables
there’s a theory of human scale networking that you can avoid cables by combining batteries and wireless networking. the first problem is that some of your pieces (smart glasses) need more bandwidth and lower latency than wireless gets you, not to mention more battery than is comfortable to wear in a headset. the second problem is you then need to charge all the pieces individually, so you end up with the weight of cables in addition to the batteries.
ditching the batteries in favor of cables only has its own problems though. the biggest one is that now the cables are subject to dynamic loads. cables suck at dynamic loads - especially the connectors.
wireless power distribution isn’t really a good plan on the human scale. and that still doesn’t solve the bandwidth issues.
so what’s the secret cable that is:
- high bandwidth
- can carry enough power (not actually that much)
- can be subject to repetitive dynamic loads
- is comfortable next to the skin
3 and 4 are less of a problem for the display, because it’s a pretty straight shot to the head from a backpack or belt. but they are very important for any hand or leg mounted hardware. on the other hand - the leg and hand mounted hardware could be low power enough (and are definitely low bandwidth enough) to do wireless only.