After acquiring rights to host uk-postcodes.com, the service based on an open source project written by @pezholio, we are pleased to announce some new and exciting changes to mark the end of the first month of this new open source partnership. With over 2.2 million requests in November, our first month of hosting the site, we’re ready to give the site a makeover!

UK-Postcodes.com is a high traffic, high availability site, responding to over 54,000 requests each day. As UK-Postcodes is a freely available, advert free service built on an open source platform it requires affordable overheads while maintaining an effective and efficient service, allowing it to cope with the high levels of demand with the lowest resource footprint. To do this Bluelight Studios runs the service on a highly specialised version of PHP, known as a Fast Process Manager.

PHP-FPM is the newest state-of-the-art technology in the PHP world. Its highly versatile configuration allow careful management of different pools of applications which helps to prevent traffic peaks from affecting the overall service. Furthermore, because the technology is run as an always-on background service, rather than opened on request, the time taken to process a request is reduced by seconds, as the system does not have any start up procedures to attend to, and also can maintain parts of the code in memory between requests. This sharing of relevant information between requests again reduces the system load, and maintains the small footprint required for financial viability. Before the service was moved across our engineers extensively tested this custom platform, after which they performed a hot transfer from the previous host, resulting in a staged migration to prevent downtime.

Within the first few days of running the stable code base of uk-postcodes.com on the platform our developers began a process of code review, using advanced tracing techniques to monitor the efficiency of the PHP application and the database subsystems involved behind the scenes. Through testing in a simulated high load development environment, our engineers found that one of the most time consuming requests was the reverse geolocation, which had the database using vital resources in its result sorting subsystem due to an inefficient query, delaying the requests. Through a combined effort with @pezholio were able to respond quickly and submit open source patches, which after more testing were swiftly adopted onto the production systems, improving request time by up to 3 seconds and decreasing the server load by 1/3!

As part of the initial commitment to UK-Postcodes, Bluelight Studios had planned a complete overhaul, adding new datasets and a customisable output feed but, after discussions with our user base we have found that our clients use this resource because of its simplicity and ease of use. For this reason the new version of UK-Postcodes will not see a vast change in functionality, but instead will see a simple theme change, branding the site with the rest of our projects. Bluelight Studios has however started work on a separate data platform, which will allow mapping, tabling and charting functionality using a flowchart style editor which we hope to be available soon. This is part of our continued commitment to helping create value from raw data provided by the open data community.

We hope you like the service and feel it is efficient. Futhermore, if you do have an open data project that you wish us to invest, or requires specialist hosting features, please get in contact with us!

Thanks also to the following contributors

Edd Robinson – Python Library
Stef Lewandowski – Ruby Gem (pat)
Stuart Harrison – PHP Function
P.S. A quick random fact: so far in this first week in December the most hit 404 page is “/latlng/32.832786,undefined.json” – at 29,168 requests! Although that made some very busy days for our servers, the service remained available as normal with no specific configuration adjustments needed.

P.P.S. We forgot to thank My Society for their MapIt API which the service relied upon for some of the electoral boundaries. Thanks Guys! We love your work!


Add Comment | Categories: News Blog | Posted: December 28, 2012

Leave a Reply

Your email address will not be published. Required fields are marked *


7 − = one

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

  • Will Tinsdeall says:

    Just an update for everyone:

    We’ve had a few people who have been making invalid requests incredibly frequently – in fact simultaneously, degrading the service for everyone. For this reason we’ve added a “soft” limit on the number of simultaneous requests to 10. How does this work?

    Well, when you get to 10 simultaneous requests, we “drop” connection packets. On a TCP/IP protocol, under which HTTP works, these will just be resent later, but will have the affect of slowing down the number of requests coming from your server without queuing them on our infrastructure.

    Please note there is still no limit to the number of requests per month. Also, if you wish to have more than 10 simultaneous connections, please contact us, and we will make provisions!

    P.S. We’re now at double the number of requests since this article was written, averaging at 105,000 per day!