A full review of EasyEDA: A circuit EDA online tool

1easyeda

Yahya Tawil wrote in to share his review of a web based EDA tool, the EasyEDA. He explains the general structure of this tool and some of its cool features:

EDA cloud tools which are related to electronics are emerging exponentially in almost all aspects (i.e. simulation, PCB design, footprint creation, gerber files viewing and 3D PCB viewing). Even well-known desktop programs like EAGLE CAD and Altium are trying to compete in this field by making their own services or by acquiring others.
Web-based EDA tool suites like EasyEDA and Upverter are getting rapidly famous. These online tools offer some outstanding solutions for collaboration and providing some viable features for teams with financial limits like multi-layer PCB designing, while it costs a lot to buy a licence for other EDA tool with a multi-layer feature, for example.

More details at allaboutcircuits.com.

CHINA STUFF: Email that works…

mail-dp

Working from China is a constant reminder that the location of internet services is super important. North America and Europe are just a few internet hops from our servers in Hetzner’s carbon neutral, green energy data center in Germany. From China, however, we’re often routed out through Beijing to San Jose, then to New York, London, Amsterdam, and finally Frankfurt. On a good day we can do that with only 2-3% packet loss.

Back in the good old days Gmail’s imap service worked perfectly even though Google was blocked, but eventually email was blocked too. At that point we added a managed mail server at Hetzner in Germany. Despite having our own non-blocked mail server, Hetzner’s data centers are unreachable during peak periods of the day because of peering agreements, congestion, geolocation, and a million other factors not worth arguing about.

The image above is a WinMTR report, a combination of ping and traceroute that helps locate network problems. 219.* is where we exit on China Unicom’s cable in Beijing and connect at San Jose, California. Nearly 50% packet loss, average ping time of 482ms. This is better than normal because San Jose connected directly to Frankfurt when this test was taken, instead of the more typical New York-> London-> Amsterdam route.

So how bad is it exactly? 5 tries to send a simple text email without attachments. “Failed to save draft, try again?” prompts every few seconds. Ability to see mail subject headers, but not download the message text. Want to send or receive an attachment? Better schedule that for late at night or first thing in the morning.

International bandwidth from China

There are three cable landings in China: Beijing, Shanghai, and Guangzhou. Each is dominated by one of the three state-owned ISPs. China Unicom (our provider) is big in the north with international traffic routing through Beijing and around 1TB/s bandwidth. China Telecom is big in the south (e.g. Shenzhen area) with around 2.5TB/s of bandwidth through Guangzhou. There are also a lot of bit players like Great Wall and TopWay that have city and regional backbones that eventually dump onto the major state-owned ISP infrastructure.

Part of our problem is using a northern ISP (Unicom) in the south. All of our traffic is routed to Beijing before leaving China. While we can see Hong Kong from the office and can walk there in about 15 minutes, connections to Hong Kong websites are routed up to Beijing and back for a 3000km+ journey. This isn’t our choice, Unicom has a monopoly in our Huaqiangbei office and nothing else is available.

This was intended to be an epic post using MTR to analyze the optimal routes and geographical locations to stash internet services with the best chance to be reachable from China. That idea bombed because nearly everything changed dramatically day to day. A test from yesterday is different than a test today, which will probably be different than a test tomorrow. Follow below for tests of several major email providers and their accessibility from China.

Rackspace Business Email is a disaster from China

rackspace

Over the past few months we tested a lot of email services. Rackspace is a well respected company with business email hosting for $2 per box per month. Signup from China triggered a review, so we had to call support to complete the order. The Rackspace rep volunteered that they have constant complaints from users in China, not something you want to hear. The MTR report shows why, routing to their imap server is a disaster (30% loss, 361ms average ping).

Microsoft Office365 email works great from China, but is itself a disaster

office365

Probably the most distressing part of daily life in China is using Microsoft’s Bing search engine. It’s a terrible search engine, but it’s always super fast within China. You might even find what you’re looking for if you skip directly to the third page of results, link number 5…

Microsoft’s Office365 imap mail service also works very well inside China, and at $4 per box per month it isn’t very expensive. The MTR report suggests Microsoft is running a server in Hong Kong that connects directly to China Unicom (219.*). The service is fast (21ms ping) and very accessible (very few lost packets).

(Un)fortunately Microsoft’s automatic email migration tool sprayed crap all over the place. Their suggestion was to hire an authorized partner for support. After a while all the subscription plans, conditions, and lack of support started to seem quite sleazy. We canceled when mail migration failed and they demanded 1 year commitments for each test account to help debug it.

ABCHK.net is our email hero

abchk

ABCHK.net is a hosting and email provider located in Hong Kong that specializes in email service for Mainland China. We were super skeptical that we could get a stable connection to Hong Kong, but they provided a test account that blew everyone away. 10 meg attachments? Uploaded and sent in seconds. It works perfectly at all times of the day, and the MTR report shows a direct, clean route to the imap server. No packets to destination lost, 31ms average ping.

For less than $7/month we get five mail boxes and 100GB of storage. Real people answered emails and handled the mail migration from our old server in Germany. After a month we are still extremely thrilled to be able to use email “normally” from inside China. Attachments upload and download super fast, and the server is always 100% reachable.

This applies to Unicom only!

Tests were done on commercial and residential China Unicom 100Mbps fiber connections. China Unicom is not the optimal ISP for Shenzhen though. It makes more sense to be on China Telecom with 2.5x more international bandwidth exiting just an hour north in Guangzhou. In the office Unicom has a monopoly, but we had a Telecom connection installed at home and will run the tests again on Telecom in a few days. Anecdotal evidence from other Telecom users doesn’t seem particularly promising though.

Admittedly this is all niche info, but it was hard won and seemed worth sharing here. At the very least someone in a similar situation might find this on Google. Or on Bing, page 3, result 5.

How to measure laser cut length or time?

800px-Laser_cutting_snowfla

To price laser cutting online we need to extract two pieces of info from uploaded design files. The area of a box surrounding the design, called the bounding box, is the amount of material needed. The length of all the cuts in the design can be used as a rough estimate of the cutting time consumed. Here’s where we’re at…

A dirty cheap laser cut service has been “coming very soon” for far too long. We planned to take SVG files and use a friend’s script to measure the bounding box and cut lengths. Unfortunately we never found a laser cut supplier willing to take low volume/high-mix orders and provide all the pricing info to give instant quotes.

Last week a comment at Hack a Day mentioned that dirty laser cutting is taking forever. Surprised by the interest, we asked around and found some willing suppliers. However, these suppliers only accept dxf, dwg, pdf, eps, and ps formats. SVG won’t do. Now we have a supplier but no way to give instant online quotes.

Conversion

Converting to and from SVG is plagued with wrecked arcs and curves. Potential customer dissatisfaction factor is super high. Absolutely not reliable enough for a commercial service or even for pricing.

Extracting info from CAD programs

Inkscape loads all the above formats, but we were not able to extract total path length from the command line or the shell interface. Autocad.io is an online version of AutoCAD that might be able to provide the design stats and a nice rendering, but if the API is slow or busy are customers supposed to wait minutes for their quotes?

A couple CAD programs seemed promising, but eventually we couldn’t find anything that runs on Linux and can extract the stats we need via the command line.

Measuring scripts

The script that measures PCBs in our store and at DirtyPCBs combines a bit of gerber code knowledge with some math to crank out basic stats without ever loading a CAD program. A small script to extract path length and bounding box size from dxf is very doable after a quick thumb through the spec, however nothing seems to exist online.

The learning curve to implement the dxf format and the debug time to perfect it puts this in the labor of love category. It would be a blast if we really wanted to do it, but since it’s standing in the way of a dirt cheap service that will likely never turn a profit (laser cutting is cheap and common…) there’s no resources to dedicate to writing our own.

Create g-code and simulate the cut

G-code is a popular way to control CNC mills and 3D printers. A CAM processor (yes, like the one in Cadsoft Eagle) combines information about the machine, such as cut speed, with the path in the design file to create a series of cutting steps or instructions. G-code is accepted universally, much like gerber files.

Lots of existing scripts convert dxf/pdf/eps/ps files to g-code. Once the design is in g-code there are several programs that simulate the cut and estimate the cut time (or filament used, etc). This seems like the proper way to do it. As a bonus, it lays the framework to calculate cut time and difficulty for a future DirtyCNC and DirtyInjectionMolding service.

This is as far as we’ve gotten. A dozen Python scripts on github look promising, but nothing directly outputs the info we need without a bit of hacking. We’ll continue to knock together a solution in the next few days.

Please shout out in the comments if you have thoughts on any of these methods or experience with any tool chains that might work. Help us get laser cutting online and we’ll reward you with – what else – free laser cutting service!

Image source: Adam Dingley CC BY SA

Amazon Web Service Beijing woefully inadequate for international use

aws-cn

Amazon Web Service (AWS) is a bunch of server, storage, and other internet resources. AWS datacenters are located around the world so companies can locate websites close to their customers and speed up page load times.

A couple years ago AWS started testing a center in Beijing, China. There’s all sorts of benchmarks on speed within and between AWS datacenters, but the Beijing location is restricted to registered Chinese companies so there’s very little public (English) info on it.

In March the internet in Shenzhen because nearly unusable. While access within China reaches 100Mbps, anything outside China experienced 50%+ packet drops and max speeds of a 20-50Kbps. In a desperate ploy we tried to setup a VPN within China to a better exit point – maybe AWS has a better connection to the outside world than our local ISPs. Keep in mind that this a VPN inside China, the internet is still filtered but is hopefully more consistent and usable.

Signing up for the AWS beta was pretty easy. The account reps were super helpful and spoke perfect English. Everything went smoothly aside from the rep’s utter disbelief that, yes, we are a real Chinese company not a Hong Kong Company, and yes, we don’t want a free trial tier, and yes, we can pay in RMB from a Chinese company bank account. We guess they get a lot of people trying to sign up for free service without a company because they mentioned these things a dozen times.

We setup a “medium instance” with Ubuntu, AWS cult code for a small server without throttling. First things first – apt-get update && apt-get upgrade – to bring the server up to date. Apt pulled from AWS repositories within the same datacenter in China at a shocking 64Kbps! They didn’t even break 100Kbps within their own datacenter! Holy crap. For comparison, we regularly FTP between our servers in Hetzner’s German datacenter at 100Mbps+. Pathetic.

Ok, what about the connection to the outside world? The vast majority of our web traffic exits China from Beijing on one of two cables: San Jose, USA and Tokyo, Japan. Surely we could take advantage of our good speeds within China and then slingshot international traffic at higher speeds through the “superior” AWS infrastructure. Not exactly. A file download from DirtyPCBs.com, located in a super high speed San Jose datacenter, topped out at 16Kbps. Really Amazon?

We did similar experiments with Aliyun, the Chinese version of AWS made by the Alibaba/Aliexpress people. The results were slightly better but still absolutely unusable as a gateway to the internet outside of China.

The quest for an internet solution continues…

Site reorganization complete

Danger-sign

Sorry for the occasional outages over the past week. Most were during Chinese work hours, so hopefully not really noticed if you’re in the US or EU.

The whole site has been moved into GIT repositories and redeployed as virtual directories as outlined on the blog last week. It is much cleaner, better organized, and more resilient against bugs and hacks.

Today the blog moved to /blog/ to make way for the DEV site to be integrated here. All old posts, pages, and images have 301 Permanent Redirects to avoid breaking links at other sites and to preserve rank with search engines. DEV will probably be moved tomorrow if all goes well. All current DEV site orders will be processed without interruption or delay, even while the site is offline.

If you’re planning to get in any super cheap PCB or SLA 3D prints be sure to do it today. When we move the store to the main site prices will increase to match DirtyPCBs.com. Once the new store is running smoothly prices at DirtyPCBs will increase to encourage people to move to the new system. The DirtyPCBs scripts weren’t designed to handle the 10s of thousands of orders you’ve put through them and they’re starting to come apart at the seams.

The new forum theme is complete. phpBB will get a big update later this week. A new MediaWiki theme and upgrade are scheduled for next week.

If you notice any problems on the blog, forum, and wiki please report it through the contact form or here in the comments.

HOW-TO: Add live websites to GIT without interruptions or security risks

2016-05-03_1636

Over at the DEV site we’ve been using GIT repositories hosted at BitBucket.org to push (deploy) changes to the live server instantly. It’s way slicker than uploading changed files by FTP, which has no roll back if something goes wrong. It also solves one of our China internet problems: routing to Bitbucket is much more reliable than to our servers in Germany.

It is finally time to merge the DEV site with the main Dangerous Prototypes sites – the blog got a new theme yesterday. When we started updating the rest of the site it seemed much easier and safer to put it in GIT too. There are a lot of tutorials about deploying websites with GIT, but none completely covered the process to safely put existing websites into GIT with no interruptions and maximum security.

Updating folder structure

We’re starting off with a legacy folder structure rooted in early cloud services and personal ignorance. The web root is at /var/www/. Other parts of the site (/forum, /docs) sit in subfolders of the root:

/www/ (WordPress blog)
/www/forum (phpBB forum)
/www/docs (MediaWiki wiki)

This isn’t how modern websites are structured. It’s super amateur, but it’s worked for years. We’ll change this to use Apache virtual directories when the dev site is merged with this site.

Use an external repo folder for security

“git init” would create a new repo around the entire website giving instant version control. It would also put the git config folder and files on the live web leaving a big security hole! It also binds together the five different web apps running on this server into a single giant mess.

git clone –mirror git@bitbucket.org:dangercore/wordpress.git repo-wordpress

After creating a free GIT repo at BitBucket.org called “wordpress” and setting up the SSH keys we’re really to roll. We cloned the empty repo into our home folder (~) on the server, keeping all the repo files out of the reach from the web.

cd repo-wordpress
GIT_WORK_TREE=/var/www git status

The GIT_WORK_TREE variable links the live website directory (/var/www) with the new repo in our home folder (~/repo-wordpress). The status command will show untracked files in the /var/www directory.

At this point you might think we should protect the git folders and config files with .htaccess instead of including the annoying GIT_WORK_TREE constant with every GIT command. We could, but then all changes would need to include that security for the life of the website. Blocking access is a patch against a vulnerability, better to never have the vulnerability in the first place.

.gitignore stuff

# Exclude files from the git repo
wp-content/cache/*
wp-content/media/*

# Include these files in previously blocked directories
!wp-content/media/.htaccess

#other software in root
docs/*
forum/*

Not everything belongs in our repo. 8 years of blog images and cache files take up a ton of space in a repo and we don’t need them to push code updates. We do, however, want to keep the .htaccess file in the media folder that prevents users browsing the contents of that directory.

We also want to exclude the other sites mixed into the webroot (docs/,forum/). We’ll push them into their own repos later using the same process. Put these rules in the .gitignore file in the GIT_WORK_TREE folder (/var/www for us). This is a short example .gitignore based on this, see our complete ignore files for all the sites in the forum!

GIT_WORK_TREE=/var/www git status

Check file status again. Ignored directories should not be on the list.

Add, Commit, Push

GIT_WORK_TREE=/var/www git add .

GIT is now tracking these files.

GIT_WORK_TREE=/var/www git status

See a list of the files being tracked.

GIT_WORK_TREE=/var/www git commit -m”Initial commit”

Commit the files to the local repo. Now we have a snapshot of the site code.

GIT_WORK_TREE=/var/www git push

Push from local repo to the remote repo at BitBucket. The site snapshot is now also stored at BitBucket.

Pull hooks for automatic updates

BitBucket and GitHub have a ‘hook’ feature that loads a URL after every push to the repo. We setup BitBucket to load a “secret” webpage on the server that triggers a git pull command whenever we push an update to the master branch. ServerPilot has some more info and a nice script that we modified.

Next steps

/apps/ (Folder of GIT repos)
/apps/wordpress/ (WordPress GIT repo with notes, tools, etc)
/apps/wordpress/public/ (Live public folder accessible on the web)

We want to end up with a structure where each area of the site (/forum,/docs,/blog) is an Apache virtual directory or symlink to a public subfolder inside a git repository. Instead of serving WordPress directly from the repo (/apps/wordpress/), we serve it from a subfolder inside the repo called “public” (/apps/wordpress/public).

First, this keeps the git folder and configuration files off web without the annoying GIT_WORK_TREE variable in each command. Second, we can use the main folder for other stuff that might be handy to have in the repository but shouldn’t be public: notes, database updates, sample files and data, test tools, etc.

We’re redeploying each site area like this as the new themes are finished. When the dust is settled we’ll document the final server setup that should last well into the future.