FiveThirtyEight has a Git repo with story related data and scripts

This repository contains a selection of the data — and the data-processing scripts — behind the articles, graphics and interactives at FiveThirtyEight.

We hope you’ll use it to check our work and to create stories and visualizations of your own. The data is available under the Creative Commons Attribution 4.0 International License and the code is available under the MIT License. If you do find it useful, please let us know.

Article Date(s) Headline(s) Folder
May 13, 2016 Some People Are Too Superstitious To Have A Baby On Friday The 13th births
May 11, 2016 Sitting Presidents Give Way More Commencement Speeches Than They Used To presidential-commencement-speeches
March 30, 2016 How Trump Hacked The Media trump-news
March 10, 2016 These Are The Phrases Each GOP Candidate Repeats Most repeated-phrases-gop
Feb. 2, 2016 Iowa Teaches Pollsters To Poll Until The End poll-of-pollsters
Jan. 6, 2016 You Can’t Trust What You Read About Nutrition nutrition-studies
Dec. 28, 2015 Most Pollsters Say Their Reputations Have Worsened poll-of-pollsters
Dec. 9, 2015 A Complete Catalog Of Every Time Someone Cursed Or Bled Out In A Quentin Tarantino Movie tarantino
Nov. 20, 2015 Here’s What Your Part of America Eats On Thanksgiving thanksgiving-2015
Nov. 14, 2015 The Rise Of Religiously Inspired Terrorism In France terrorism
Nov. 5, 2015 College Students Aren’t The Only Ones Abusing Adderall study-drugs
Oct. 15, 2015 Be Suspicious Of Online Movie Ratings, Especially Fandango’s fandango
Aug. 14, 2015 ‘Straight Outta Compton’ Is The Rare Biopic Not About White Dudes biopics
Aug. 6, 2015 Every Guest Jon Stewart Ever Had On ‘The Daily Show’ daily-show-guests
July 30 22, 2015 Buster Posey’s Pitch Framing Makes Him A Potential MVP buster-posey-mvp
July 22, 2015 What 12 Months Of Record-Setting Temperatures Looks Like Across The U.S. us-weather-history
June 19, 2015 Projecting The Top 50 Players In The 2015 NBA Draft Class nba-draft-2015
June 10, 2015 The Most Common Unisex Names In America: Is Yours One Of Them? unisex-names
June 9, 2015 Blatter’s Reign At FIFA Hasn’t Helped Soccer’s Poor elo-blatter
June 5, 2015 Why Some Tennis Matches Take Forever tennis-time
June 3, 2015 Where Police Have Killed Americans In 2015 police-killings
June 2, 2015 FiveThirtyEight’s Women’s World Cup Predictions womens-world-cup-predictions
June 2, 2015 The Rock Isn’t Alone: Lots Of People Are Worried About ‘The Big One’ san-andreas
May 29, 2015 How To Break FIFA fifa
May 21, 2015 Complete History Of The NBA nba-elo
May 19, 2015 Evangelical Protestants Are The Biggest Winners When People Change Faiths pew-religions
May 15, 2015 ‘Mad Men’ Is Ending. What’s Next For The Cast? mad-men
May 12, 2015 Joining The Avengers Is As Deadly As Jumping Off A Four-Story Building avengers
April 23, 2015 How Baby Boomers Get High drug-use-by-age
April 20, 2015 Where People Go To Check The Weather weather-check
March 15, 2015 FiveThirtyEight’s 2015 NCAA Tournament Predictions march-madness-predictions-2015
Feb. 18, 2015 Every NBA Team’s Chance Of Winning In Every Minute Across Every Game nba-winprobs
Jan. 30, 2015 Who Will Run For President: Romney Is Out potential-candidates
Jan. 14, 2015 Looking For Clues: Who Is Going To Run For President In 2016? potential-candidates
Dec. 30, 2014 Who Goes To Meaningless NFL Games And Why? nfl-ticket-prices
Nov. 20, 2014 Dear Mona, What’s The Most Common Name In America? most-common-name
Nov. 7, 2014 Most Pollsters We Surveyed Say Polling Did Well This Year poll-of-pollsters
Oct. 31, 2014 Even Pollsters Don’t Know All The Details Of How Their Polls Are Made poll-of-pollsters
Oct. 20, 2014 Pollsters Say They Follow Ethical Standards, But They Aren’t So Sure About Their Peers poll-of-pollsters
Oct. 13, 2014 Comic Books Are Still Made By Men, For Men And About Men comic-characters
Oct. 6, 2014 Pollsters Predict Greater Polling Error In Midterm Elections poll-of-pollsters
Oct. 3, 2014 Dear Mona, How Many Flight Attendants Are Men? male-flight-attendants
Sept. 29, 2014 Marriage Isn’t Dead — Yet marriage
Sept. 25, 2014 FiveThirtyEight’s Pollster Ratings pollster-ratings
Sept. 25, 2014 How FiveThirtyEight Calculates Pollster Ratings pollster-ratings
Sept. 17, 2014 How The FiveThirtyEight Senate Forecast Model Works forecast-methodology
Sept. 12, 2014 The Economic Guide To Picking A College Major college-majors
Sept. 9, 2014 The NFL’s Uneven History Of Punishing Domestic Violence nfl-suspensions
Sept. 5, 2014 41 Percent of Fliers Say It’s Rude To Recline Your Airplane Seat flying-etiquette-survey
Aug. 20, 2014 Most Police Don’t Live In The Cities They Serve police-locals
Aug. 13, 2014 Dear Mona Followup: Where Do People Drink The Most Beer, Wine And Spirits? alcohol-consumption
Aug. 4, 2014 The Football Hall Of Fame Has A Receiver Problem nfl-wide-receivers
July 25, 2014 Dear Mona, How Many Couples Sleep in Separate Beds? sleeping-alone-data
July 22, 2014 America’s Favorite ‘Star Wars’ Movies (And Least Favorite Characters) star-wars-survey
July 18, 2014 Should Travelers Avoid Flying Airlines That Have Had Crashes in the Past? airline-safety
June 25, 2014 The FiveThirtyEight International Food Association’s 2014 World Cup food-world-cup
June 17, 2014 Elitist, Superfluous, Or Popular? We Polled Americans on the Oxford Comma comma-survey-data
June 9, 2014 FiveThirtyEight’s World Cup Predictions world-cup-predictions
June 6, 2014 What Ethan Swan Learned From Tracking Every Tattoo in the NBA nba-tattoos
June 3, 2014 Using Infrastructure Jobs as a Measuring Stick For State-Level Spending infrastructure-jobs
April 29-30, 2014 Which States Are in the South? / Which States Are in the Midwest? region-survey
April 28, 2014 Both Republicans And Democrats Have an Age Problem congress-age
April 16, 2014 Early Senate Polls Have Plenty to Tell Us About November early-senate-polls
April 15, 2014 Do Pulitzers Help Newspapers Keep Readers? pulitzer
April 14, 2014 A Statistical Analysis of the Work of Bob Ross bob-ross
April 11, 2014 Where Are America’s Librarians? librarians
April 1, 2014 The Dollar-And-Cents Case Against Hollywood’s Exclusion of Women bechdel
March 27, 2014 The NCAA Bracket: Checking Our Work historical-ncaa-forecasts
March 17, 2014 FiveThirtyEight’s NCAA Tournament Predictions march-madness-predictions

Original URL:  

Original article

Want to spell check? Read the fine print

Posted on May 30 2016

Update: This post has hit Hacker News – you might want to follow the comments over there.

I have been playing around with the very nice Visual Studio Code editor recently. For those who don’t know, it’s a free source code editor from Microsoft which works cross-platform. The reason I was giving it a look is that it has pretty good support for editing Go, a language I am trying to find the time to learn. I was also keen to see what the New Microsoft could offer a non-microsofty like myself.

I like what I’ve seen so far, but something interesting came up today. I realised that unlike Sublime, Visual Studio Code wasn’t doing any spell checking out of the box. No problem I thought, I bet there is an extension for that. And guess what, there is:

The Spelling and Grammar Checker for Visual Studio Code

Visual Studio gives you a nice browsing interface to find likely packages, and even get a sense of how popular they are. The clear winner here is Sean McBreen’s Spelling and Grammar Checker. You can also see in the above screen shot that you get a link to details about the extension. As I was still pretty new to the ecosystem, I was sort of interested as to what information you got about extensions, and whether or not I’d found the spell check plugin for Visual Studio Code, or just a plugin. Clicking through…

The Spelling and Grammar Checker for Visual Studio Code

Initially, things are promising. A lot of installs (compared to other extensions) and a bunch of 5 star reviews:

After The Deadline – a Spell Checker for WordPress (and other things)

But then, at the top of the description, I found this message greeting me:

Notice: This extension uses the teacher node module which calls the After The Deadline service to check for spelling and grammatical errors. Document text is sent to the service over unencrypted HTTP. Do not use this extension with sensitive or private documents.

So to be clear, this is saying that any text opened in Visual Studio Code with this extension loaded would be sent in plain text to some service I’ve never heard of. The mind boggles at how terrible this is as an idea for an editor designed for source code editing. I wasn’t the only person who thought so:

Is This Really True?

My first reaction was “Surely I’ve misunderstood something, right?”. So I enabled the extension and opened up a non-sensitive file, one of my in-progress blog posts. I then downloaded wireshark to take a look. It took a while to sift it out – it’s been a LONG time since I used wireshark (so long ago that back then it had still been called Ethereal), and there is a rather large amount of traffic generated from my machine for things like Sonos and Dropbox and the like, but eventually I tracked down what was being sent. Sure enough I could see all the text being sent, unencrypted, over HTTP to the ‘After The Deadline’ service. So, at least the documentation was accurate, and that this absolutely insane thing was happening.

What’s Going On Here?

As per the documentation, the extension is just making use of a node plugin called teacher, which is just a thin node API (last updated over 4 years ago) to help call a service called After The Deadline. This piqued my interest – who were the people behind this service? Could they be trusted?

After The Deadline – a Spell Checker for WordPress (and other things)

Their website immediately made me think of some of the pretty bad stock product website from 2008, but I was surprised to see that the company behind the service is Automattic, the company behind WordPress, amongst other things. This started to make a bit more sense. In the context of a blog post, which is something designed to be shared publicly, the fact that this information is being sent to another party isn’t too much of a problem. But in the context of the sorts of things we open up in a source code editor, this is utterly insane.

It’s About Trust, Right?

Partly, the problem here is that this traffic is being sent unencrypted. We’ll get to that in a moment. But even if this service did use HTTPS (and I can see no good reason why it can’t), by using this plugin I am explicitly trusting that not only with the service I send my data to not do anything silly with the information they receive, but that they will ensure it is properly managed and not left lying around to be swept up by some malicious attacker. And again, the amount of sensitive information this could include could be staggering. Source code can be bad enough, but think of the number of times you’ve put sensitive things like AWS keys or passwords into text files. Now think what happens if they fall into the wrong hands.

The HTTP-only support is also nuts. This means that even if I do trust Automattic to look after my potentially private and sensitive information, I’m opening myself up to people pretending to be After The Deadline. Given the use of a very old node plugin in this tool chain, I did wonder if perhaps After The Deadline did support HTTPS, but that the extension’s reliance on the teacher plugin may have limited the ability to use a slightly more sensible protocol. Poking around, it becomes pretty clear that After The Deadline is a freely available service, which Automattic have decided to run for you. You are free to run your own copy of the stack if you want, which you could then decide to protect with HTTPS. From the docs it wasn’t clear if Automattic support HTTPS on the version they host, but I couldn’t find anything in the FAQ or documentation to imply that they did.

Update: Thanks to a comment over on Hacker News from zerocrates, it seems that After The Deadline does in fact support HTTPS in addition to HTTP (although I think my point about this not being clear in the docs still stands). Another commenter points out a recently opened issue against the teacher module is asking for HTTPS support.

What Should Happen?

Just so we’re clear, a sensible solution, for a spell checker that needs to run over sensitive information, is that it should run locally (ideally using a system & user dictionary), without any remote calls needed. It seems odd that I have to spell that out (pardon the pun) but the fact that this extension exists at all seems to be a good enough reason to have to call this out. And it turns out that popularity doesn’t always steer us in the right direction – the second spell checking extension that shows up and has a fraction of the installs (88 vs 20K), is a good old fashioned checker that uses a local dictionary. Go use that one instead.

Alternatively of course, we could wait for another party to help us here:

Automattic should give serious thought to supporting their freely available service over HTTPS too. Even though I can’t see any situation where I’d want to use this service in this specific context, for the WordPress ecosystem having this service served over a sensible, and in this day and age easy to implement protocol, would be a very good idea. Update: As pointed out above, it seems that After The Deadline can support HTTPS.

I’m sure that Sean McBreen, who wrote this plugin, has no malicious intent. He spent his own time creating this extension, and gave back to the community. He did a great job implementing and documenting an extension that I think is unfortunately misguided in this particular context. I’m going to try and track down Sean to see if he has any views on this (the marketplace for extensions doesn’t seem to give you an obvious way to find or contact authors, which seems an oversight). To be fair to him, he’s been extremely clear in the documentation as to what happens under the hood, but I’m still surprised that the use of an external (and un-trustable) service in the context of code is sensible. I’m just as surprised by the number of 5 star reviewers (and the 20K+ installs) listed on the marketplace – perhaps the reviewers have a different view of security to me.

My advice, in the strongest possible terms, is to not use this extension, unless the only things you will ever open with Visual Studio Code are things that you don’t mind being viewed by completely unknown people. Oh, and always read the fine print before installing extensions in the future. I know I will…

Back to Blog.

Original URL:  

Original article

How and Why to Log Your Bash History

For the last three and a half years, every single command I’ve run from the command line on my MacBook Pro has been logged to a set of log files.

Uncompressed, these files take up 16 MB of disk space on my laptop. But the return I’ve gotten on that small investment is immense. Being able to go back and find any command you’ve run in the past is so valuable, and it’s so easy to configure, you should definitely set it up today. I’m going to share how to do this so you can take advantage of it as well.

Bash Configuration File

You’ll need to configure an environment variable so that it’s loaded in every command line session. On my MacBook Pro, I use the .bash_profile file. On other operating systems, the .bashrc file is an option. See this blog post on .bash_profile vs .bashrc for more on the differences.


The Bash Prompt HOWTO describes the PROMPT_COMMAND environment variable as follows:

Bash provides an environment variable called PROMPT_COMMAND. The contents of this variable are executed as a regular Bash command just before Bash displays a prompt.

We’re going to set the PROMPT_COMMAND variable to be something that logs the most recent line of history to a file. To do this, add the following to your chosen Bash configuration file (.bash_profile for me):

export PROMPT_COMMAND='if [ "$(id -u)" -ne 0 ]; then echo "$(date "+%Y-%m-%d.%H:%M:%S") $(pwd) $(history 1)" >> ~/.logs/bash-history-$(date "+%Y-%m-%d").log; fi'

First, this checks to make sure we’re not root.

If that checks out, it appends a line that includes the current timestamp, the current working directory, and the last command executed to a log file that includes the current date in the filename.

Having the commands stored in separate files like this really helps when you’re trying to find a command you ran sometime last month, for example.

> grep -h logcat ~/.logs/bash-history-2016-04*
2016-04-01.10:18:03 /Users/me 66555  adb logcat
2016-04-01.10:19:56 /Users/me 66555  adb logcat
2016-04-01.11:01:36 /Users/me 66555  adb logcat
2016-04-05.09:50:25 /Users/me/git/android-project 66368  adb logcat
2016-04-05.13:42:54 /Users/me/git/android-project 66349  adb -s emulator-5554 logcat
2016-04-06.10:40:08 /Users/me/git/android-project 66390  adb logcat
2016-04-06.10:48:54 /Users/me/git/android-project 66342  adb logcat


It will only take a few seconds to update your PROMPT_COMMAND so that it logs every command to a file.

And the next time you’re trying to remember the command line options you used with find that one time (but can’t find in your current session’s history), you’ll be able to look it up in the log files.

Oh, and if you want to know how many times you’ve done a git push in the last three and a half years, you can look that up, too (5,585 git pushes for me)!

Original URL:  

Original article

DeepOSM: Detect roads and features in imagery with neural nets using OpenStreetMap

Classify roads and features in satellite imagery, by training neural networks with OpenStreetMap (OSM) data. DeepOSM lets you:

  • Download a chunk of satellite imagery
  • Download OSM data that shows roads/features for that area
  • Generate training and evaluation data
  • Display predictions of mis-registered roads in OSM data, or display raw predictions of ON/OFF

Running the code is as easy as install Docker, make dev, and run a script.

Contributions are welcome. Open an issue if you want to discuss something to do, or email me.

Default Data/Accuracy

By default, DeepOSM will analyze about 200 sq. km of area in Delaware. DeepOSM will

  • predict if the center 9px of a 64px tile contains road.
  • use the infrared (IR) band and RGB bands.
  • be 75-80% accurate overall, training only for a minute or so.
  • use a single fully-connected relu layer in TensorFlow.
  • render, as JPEGs, “false positive” predictions in the OSM data – i.e. where OSM lists a road, but DeepOSM thinks there isn’t one.

NAIP with Ways and Predictions

Background on Data – NAIPs and OSM PBF

For training data, DeepOSM cuts tiles out of NAIP images, which provide 1-meter-per-pixel resolution, with RGB+infrared data bands.

For training labels, DeepOSM uses PBF extracts of OSM data, which contain features/ways in binary format that can be munged with Python.

The NAIPs come from a requester pays bucket on S3 set up by Mapbox, and the OSM extracts come from geofabrik.

Install Requirements

DeepOSM has been run successfully on both Mac (10.x) and Linux (14.04 and 16.04). You need at least 4GB of memory.

AWS Credentials

You need AWS credentials to download NAIPs from an S3 requester-pays bucket. This only costs a few cents for a bunch of images, but you need a credit card on file.


Install Docker

First, install a Docker Binary.

I also needed to set my VirtualBox default memory to 4GB, when running on a Mac. This is easy:

  • start Docker, per the install instructions
  • stop Docker
  • open VirtualBox, and increase the memory of the VM Docker made

Run Scripts

Start Docker, then run:

Download NAIP, PBF, and Analyze

Inside Docker, the following Python scripts will work. This will download all source data, tile it into training/test data and labels, train the neural net, and generate image and text output.

The default data is six NAIPs, which get tiled into 64x64x4 bands of data (RGB-IR bands). The training labels derive from PBF files that overlap the NAIPs.

python bin/
python bin/

For output, DeepOSM will produce some console logs, and then JPEGs of the ways, labels, and predictions overlaid on the tiff.

Jupyter Notebook

Alternately, development/research can be done via jupyter notebooks:

To access the notebook via a browser on your host machine, find the IP VirtualBox is giving your default docker container by running:

docker-machine ls

NAME      ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER    ERRORS
default   *        virtualbox   Running   tcp://           v1.10.3

The notebook server is accessible via port 8888, so in this case you’d go to:


Also see a work journal here.

Papers – Relevant Maybe

Papers – Not All that Relevant

Papers to Review

Recent Recommendations

Citing Mnih and Hinton

I am reviewing these papers from Google Scholar that both cite the key papers and seem relevant to the topic.

Original Idea

This was the general idea to start, and working with TMS tiles sort of worked (see first 50 or so commits), so DeepOSM got switched to better data:

Deep OSM Project

Original URL:  

Original article

Open-source hardware PlayStation GL renderer – alpha version

Parasite Eve with the experimental GL renderer. Note the screen residue glitches at the bottom.
Parasite Eve with the experimental GL renderer. Note the screen residue glitches at the bottom.

Here is the first release of Mednafen/Beetle PSX HW. You can download this core right now for Windows, Linux, and OSX.


See a complete screenshot gallery here.

Where to get it?

Rage Racer using experimental GL renderer
Rage Racer using experimental GL renderer
Final Fantasy VII using experimental GL renderer
Final Fantasy VII using experimental GL renderer

Go to RetroArch, go to Online Updater, go to ‘Update Cores’, and download ‘PlayStation (Mednafen PSX HW)’. You might need to update your core info files first before this will show up properly. To do this, go to ‘Online Updater’, and select Update Core Info Files’.

System requirements

You need to have a GPU that supports at least OpenGL 3.3 core context in order to use this. It might be possible to lower GPU requirements in time but now is not yet that time.


Gran Turismo 1 with the experimental GL renderer.
Gran Turismo 1 with the experimental GL renderer.

We have added an OpenGL renderer to Mednafen PSX. This renderer is based on simias’ GPU renderer for his Rust-based PlayStation emulator, Rustation.

Lots of people worked hard on bringing this to you. It was a joint team effort between r5z, simias, zeromus, and me to bring this from Rust over to C++, and in a way so that it still benefits Rustation as well.

This is probably the first OpenGL renderer for PSX that is not based on outdated fixed-function OpenGL code and which is, actually, well, open-source. Because we can assume shaders will be there, we can get reasonably close to emulating texture windows accurately (with Pete’s OGL1 GPU plugin, there were codepaths for doing it with paletted textures – the hacky approach – and the pixel shader approach). Therefore, the renderer is still kept pretty simple and easy to understand.

Unfinished business

Resident Evil 2 with the experimental GL renderer.
Resident Evil 2 with the experimental GL renderer.

There are several parts which are still not finished:

    VRAM-to-VRAM copies right now are completely broken. You might see several glitches during FMVs and/or other scenes that require framebuffer manipulation. Final Fantasy 7/8’s pre-battle swirls come to mind as examples of that.
    There are still several texture updating problems. The water in Tomb Raider 2 for instance can initially appear flat shaded until you take a savestate. Other problems include Resident Evil 3’s sprite-based inventory system, when you add or remove items from this list, the changes might not be reflected on-screen.
    There are still some graphics issues. One example is the broken opacity boxes around Silent Hill’s objects.
    There are still several crippling performance bottlenecks with this GL renderer. The first priority is getting the rendering to be as accurate as the software renderer until optimization becomes a priority.
    Toggling fullscreen will give you a black screen. This is yet to be implemented.
    There are currently some near/far issues right now where objects very close to the projection camera can appear glitched onscreen.
    There might still be some resolution issues where some portions of the screen don’t get clipped away as they should, so you might see some onscreen garbage in places that are normally obscured and not visible.


Picture of Tomb Raider with nearest point filtering
Picture of Tomb Raider with 3 point N64 filtering
Picture of Tomb Raider with bilinear filtering

Let’s detail some options for this hardware-based core :

Internal GPU Resolution (Restart) – This lets you change the internal resolution. NOTE: In this alpha version, it will crash if you try to change this at runtime. Here is a way to work around it: go to Quick Menu -> Options, change ‘Renderer’ to ‘software’, restart the core. Change the Internal GPU Resolution. Then change ‘Renderer’ back to ‘opengl’, and restart. Your changes should take effect now. Right now the max resolution you can set it to is 8x.
Texture filtering – This only works with the OpenGL renderer. You can toggle between ‘nearest’ point filtering, 3-point filtering (N64-style quasi-linear filtering), and plain bilinear filtering. Both 3-point filtering and bilinear filtering can impact performance negatively right now, and on average bilinear filtering has less graphics issues than 3-point so far. If you want the best performance, go with ‘nearest’.
Internal color depth – This only works with the OpenGL renderer. You can toggle between 32bpp (32bit color) and 16bpp dithered mode.
Wireframe mode – It will toggle between wireframe mode and filled shading mode (the default).
Display full VRAM – This requires a restart in order to take effect. This allows you to see all the VRAM banks on-screen. Mainly useful for developers only.

Warning for people using Cg shaders

Street Fighter Ex Plus Alpha with the experimental GL renderer and bilinear filtering enabled.
Street Fighter Ex Plus Alpha with the experimental GL renderer and bilinear filtering enabled.

Cg shaders won’t work with this core as long as it requires OpenGL 3.3 core context. Cg is no longer alive from a public consumption perspective and as such they never saw fit to adding support for GL 3.3 core context.

So make sure to disable any Cg shaders you had running before trying to run this core.

When/if we lower GL requirements, this might no longer become an issue, but you will always have a better experience with this plugin using GL 3.3 or later.

Bug reports

Tekken 3 with experimental GL renderer
Tekken 3 with experimental GL renderer

You can post bugs and graphics issues here (preferably with screenshots).

Other Mednafen/Beetle PSX improvements

Final Fantasy VIII with experimental GL renderer
Final Fantasy VIII with experimental GL renderer

PBP support is now much more mature. There is support for encrypted official PS3/PSP PSOne games added now thanks to Zapeth.

He also submitted some accuracy improvements which should give the emulator a better score with the PSX benchmark test programs.

Upcoming news/announcements

Metal Gear Solid
Metal Gear Solid

In other news, expect a new point release of RetroArch soon. No Apple TV and Windows UWP support yet, but Blackberry 10 might be coming back for the 1.3.5 point release.

Original URL:  

Original article

The app to nowhere

bridgeout Smartphone apps have improved convenience for public transportation in many urban centers. In Washington, DC, riders can download apps to help them figure out where to go, when to show up and how long to wait for a bus or train. However, the problem with public transport in DC is not the lack of modern, helpful and timely information. The problem is that the Metro subway system is on fire. Read More

Original URL:  

Original article

HyperDev – Developer Playground for Full-Stack Web Apps

by Joel Spolsky

Tuesday, May 31, 2016

One more thing…

It’s been awhile since we launched a whole new product at Fog Creek Software (the last one was Trello, and that’s doing pretty well). Today we’re announcing the public beta of HyperDev, a developer playground for building full-stack web-apps fast.

HyperDev is going to be the fastest way to bang out code and get it running on the internet. We want to eliminate 100% of the complicated administrative details around getting code up and running on a website. The best way to explain that is with a little tour.

Step one. You go to

Boom. Your new website is already running. You have your own private virtual machine (well, really it’s a container but you don’t have to care about that or know what that means) running on the internet at its own, custom URL which you can already give people and they can already go to it and see the simple code we started you out with.

All that happened just because you went to

Notice what you DIDN’T do.

  • You didn’t make an account.
  • You didn’t use Git. Or any version control, really.
  • You didn’t deal with name servers.
  • You didn’t sign up with a hosting provider.
  • You didn’t provision a server.
  • You didn’t install an operating system or a LAMP stack or Node or operating systems or anything.
  • You didn’t configure the server.
  • You didn’t figure out how to integrate and deploy your code.

You just went to Try it now!

What do you see in your browser?

Well, you’re seeing a basic IDE. There’s a little button that says SHOW and when you click on that, another browser window opens up showing you your website as it appears to the world. Notice that we invented a unique name for you.

Over there in the IDE, in the bottom left, you see some client side files. One of them is called index.html. You know what to do, right? Click on index.html and make a couple of changes to the text.

Now here’s something that is already a little bit magic… As you type changes into the IDE, without saving, those changes are deploying to your new web server and we’re refreshing the web browser for you, so those changes are appearing almost instantly, both in your browser and for anyone else on the internet visiting your URL.

Again, notice what you DIDN’T do:

  • You didn’t hit a “save” button.
  • You didn’t commit to Git.
  • You didn’t push.
  • You didn’t run a deployment script.
  • You didn’t restart the web server.
  • You didn’t refresh the page on your web browser.

You just typed some changes and BOOM they appeared.

OK, so far so good. That’s a little bit like jsFiddle or Stack Overflow snippets, right? NBD.

But let’s look around the IDE some more. In the top left, you see some server side files. These are actual code that actually runs on the actual (virtual) server that we’re running for you. It’s running node. If you go into the server.js file you see a bunch of JavaScript. Now change something there, and watch your window over on the right.

Magic again… the changes you are making to the server-side Javascript code are already deployed and they’re already showing up live in the web browser you’re pointing at your URL.

Literally every change you make is instantly saved, uploaded to the server, the server is restarted with the new code, and your browser is refreshed, all within half a second. So now your server-side code changes are instantly deployed, and once again, notice that you didn’t:

  • Save
  • Do Git incantations
  • Deploy
  • Buy and configure a continuous integration solution
  • Restart anything
  • Send any SIGHUPs

You just changed the code and it was already reflected on the live server.

Now you’re starting to get the idea of HyperDev. It’s just a SUPER FAST way to get running code up on the internet without dealing with any administrative headaches that are not related to your code.

Ok, now I think I know the next question you’re going to ask me.

“Wait a minute,” you’re going to ask. “If I’m not using Git, is this a single-developer solution?”

No. There’s an Invite button in the top left. You can use that to get a link that you give your friends. When they go to that link, they’ll be editing, live, with you, in the same documents. It’s a magical kind of team programming where everything shows up instantly, like Trello, or Google Docs. It is a magical thing to collaborate with a team of two or three or four people banging away on different parts of the code at the same time without a source control system. It’s remarkably productive; you can dive in and help each other or you can each work on different parts of the code.

“This doesn’t make sense. How is the code not permanently broken? You can’t just sync all our changes continuously!”

You’d be surprised just how well it does work, for most small teams and most simple programming projects. Listen, this is not the future of all software development. Professional software development teams will continue to use professional, robust tools like Git and that’s great. But it’s surprising how just having continuous merging and reliable Undo solves the “version control” problem for all kinds of simple coding problems. And it really does create an insanely addictive form of collaboration that supercharges your team productivity.

“What if I literally type ‘DELETE * FROM USERS’ on my way to typing ‘WHERE id=9283’, do I lose all my user data?”

Erm… yes. Don’t do that. This doesn’t come up that often, to be honest, and we’re going to add the world’s simplest “branch” feature so that optionally you can have a “dev” and “live” branch, but for now, yeah, you’d be surprised at how well this works in practice even though in theory it sounds terrifying.

“Does it have to be JavaScript?”

Right now the server we gave you is running Node so today it has to be JavaScript. We’ll add other languages soon.

“What can I do with my server?”

Anything you can do in Node. You can add any package you want just by editing package.json. So literally any working JavaScript you want to cut and paste from Stack Overflow is going to work fine.

“Is my server always up?”

If you don’t use it for a while, we’ll put your server to sleep, but it will never take more than a few seconds to restart. But yes for all intents and purposes, you can treat it like a reasonably reliably, 24/7 web server. This is still a beta so don’t ask me how many 9’s. You can have all the 8’s you want.

“Why would I trust my website to you? What if you go out of business?”

There’s nothing special about the container we gave you; it’s a generic VM running Node. There’s nothing special about the way we told you to write code; we do not give you special frameworks or libraries that will lock you in. Download your source code and host it anywhere and you’re back in business.

“How are you going to make money off of this?”

Aaaaaah! why do you care!

But seriously, the current plan is to have a free version for public / open source code you don’t mind sharing with the world. If you want private code, much like private repos, there will eventually be paid plans, and we’ll have corporate and enterprise versions. For now it’s all just a beta so don’t worry too much about that!

“What is the point of this Joel?”

As developers we have fantastic sets of amazing tools for building, creating, managing, testing, and deploying our source code. They’re powerful and can do anything you might need. But they’re usually too complex and too complicated for very simple projects. Useful little bits of code never get written because you dread the administration of setting up a new dev environment, source code repo, and server. New programmers and students are overwhelmed by the complexity of distributed version control when they’re still learning to write a while loop. Apps that might solve real problems never get written because of the friction of getting started.

Our theory here is that HyperDev can remove all the barriers to getting started and building useful things, and more great things will get built.

“What now?”

Really? Just go to HyperDev and start playing!



College students: my company has paid
summer internships in
New York City
including free housing, free lunch, and the chance to develop software people
will really use, with great mentors
on interesting projects. Don’t miss this chance of a lifetime. We only have
a few spaces and they always go fast, so apply today.



Have you been wondering about Distributed Version Control? It has been a huge productivity
boon for us, so I wrote Hg Init, a Mercurial tutorial—check it out!


HyperDev is the developer playground for building full-stack web apps, fast.

Want to know more?

You’re reading Joel on Software, stuffed
with years and years of completely raving mad articles about software development, managing
software teams, designing user interfaces, running successful software companies,
and rubber duckies.

About the author.

I’m Joel Spolsky,
co-founder of Trello and
Fog Creek Software, and CEO of Stack Overflow.

More about me.

© 2000-2016 Joel Spolsky

Original URL:  

Original article

How to Install Nginx, PHP and MySQL (LEMP Stack) on OpenSUSE Leap 42.1

LEMP or Linux, Engine-x, MySQL, and PHP is a collection of software installed on the Linux operating system to get your PHP based web applications up and running on the fast Nginx web server. In this tutorial, I will show you how to install Nginx, MariaDB, and PHP-FPM on openSUSE leap 42.1. Then I will configure the OpenSUSE firewall with SuSEfirewall2 to allow access to the Nginx web server and show you how you can add a new virtual host configuration on the Nginx web server.

Original URL:  

Original article

Intel Core i7 Extreme Edition processor has 10 cores and 4 channel DDR4-2400 RAM

When it comes to building a PC, there are many directions you can go. Of course, the most important question you must ask is, what are you going to do with it? Ideally, you don’t want to pay for more than you need. If you will just be surfing the web, you probably don’t need a hardcore GPU and 32GB of RAM. With that said, some folks with money will just buy the best regardless. If you can afford it, hey, why not? Today, Intel announces its Broadwell-E line, including its fastest processor ever, the new Core i7 Extreme Edition i7-6950X.… [Continue Reading]

Original URL:  

Original article

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑

%d bloggers like this: