News: Operating e-Commerce Business During Regional Power Outage

Imagine if the electricity to your business was cut off. But not just your business -- your city, your county and your entire region!

That's what ameriDroid was faced with due to PG&E's Public Safety Power Shutoff that started on Saturday, October 26, 2019, and lasted until the afternoon of Wednesday, October 30 affecting our warehouse in Northern California. Read the Wikipedia article about it here.

Our Challenge

We could've just relaxed
at the beach!

ameriDroid's staff didn't want to delay the shipping of packages to our customers, and also didn't want to have customers confronted with dead phone lines when trying to call in. So we knew we couldn't just kick back in our beach chairs with tropical drinks until the power came back on!

Our Infrastructure

Because ameriDroid is a single board computer distributor, retailer, and wholesaler, we want to be intimately familiar with the products we sell. In addition to research and theory, using our products in the real-world is one of the best ways to do this. We decided from the beginning that we would operate as much as possible using only the products we sell.

We are also committed to minimizing our impact on the environment. We do this in several ways:

• We are completely paperless in our operations other than:
• cardboard boxes and packing material we use for shipping
• thermal shipping labels
• thermal packing receipts
• We recycle any excess cardboard, paper and packing material we get from other sources
• Our shipping staff use Android tablets for the majority of their work in the warehouse
• 
  Our warehouse infrastructure runs on ARM-based ODROID SBCs with one low-power embedded Intel system (for shipping peripherals that run on Windows, like digital scales and address label printers) - this allows for redundancy and low power requirements
• Our video security system runs on custom-designed Raspberry Pi camera units with an ODROID-U3 acting as a DVR, which is an extremely power-efficient system
• Our site premises security system runs on an ODROID-XU4Q and battery-powered wireless door and motion sensors, again extremely power-efficient
• Our peripheral storage facility runs on an off-grid solar array, but is too far from our warehouse to act as a power supply for the main shipping location

We employ Internet connectivity from 3 different providers for redundancy, but only one ended up being viable during the regional outage:

• Comcast - Down due to regional equipment failure
• Verizon - Marginally operational with high latency and about 50% packet loss due to overloaded networks
• Pacific.net Bonded DSL - Fully operational

Because our parent company, Respectech, provides our main infrastructure on industry-standard equipment, we had to set up a temporary network to allow ameriDroid to operate on the Pacific.net connection. Respectech's server rack takes 30A of power to operate, so it was not feasible to power these Windows- and CentOS-based servers on our off-grid solution. ameriDroid's phone systems also operate off Respectech's infrastructure. Fortunately, ameriDroid's calls automatically fail-over to our staff's mobile phones during an outage.

Our Solution

ameriBus with our mascot, Tippy

On our first day of off-grid operations, we enlisted the ameriBus to provide power with its 4000W peak pure sine wave inverter. The ameriBus has limo-style perimeter seating, so we installed this inverter to allow the ameriBus to act as a portable conference room and demo facility for our SBC solutions at fairs, conferences, and when visiting our west-coast clients. It also came in handy for our unexpected off-grid requirements.

Although this system worked perfectly well, the ameriBus consumed about one gallon of fuel for every 4 hours of idling due to the big v10 engine. We knew we could do better.

1000W pure sine wave inverter connected to the shipping van

The following day, we took a 1000W pure sine wave inverter being used on our peripheral storage facility's solar power system and connected it to ameriDroid's shipping van. The shipping van has a much smaller 6 cylinder engine. This change allowed us to operate on approximately 1 gallon of fuel for every 8 hours of operation. Because we operate on SBCs, tablets, and low-power thermal printers, 1000W of current was more than enough. An ironic challenge was that even during the day, our shipping staff had to use LED-powered headlamps to work in the warehouse as the overhead lighting wasn't able to be powered up.

The Outcome

We lost a few hours of standard productivity by having to set up off-grid power distribution solutions and building a temporary network for our shipping infrastructure. Other than that, we were nearly fully operational for the extent of the 5-day-long Public Safety Power Shutoff event, and most customer orders went out on schedule. The few that didn't make the shipping cut-off went out the very next day.

We'll be even more prepared for the next challenge, thanks to the possibilities provided by single-board computers!