Marketplace Development Last Year and Our Roadmap

A little while ago, we promised a post about our development roadmap, as well as what has been worked on in the last year. There is a laundry list of excellent features and updates that have been waiting in our pipeline for what feels like forever. And understandably there’s a lot of questioning on the forums over why they haven’t materialized yet.

Search improvements, ratings improvements, a shopping cart, item support tools, affiliate campaigns and analytics, license improvements and the list goes on. It’s a list of things that would make buying better, selling better, and alleviate problems left, right and center.
The long and the short is that a lot of these types of improvements keep getting put on hold as we cope with growth and expansion.

[collapse title=”Growth and Expansion” collapseid=”Growth”]The growth on our marketplaces has been breakneck over the last few years with everything from revenue to user count to transaction volume to item count to fraud attempts to security hacks doubling, tripling, quadrupling or more year on year. To be honest I find a lot of the numbers kind of staggering.[/collapse][collapse title=”Search & Review” collapseid=”SearchReview”]Two other areas we spent a lot of our time on last year (14% and 8% respectively) were Search and Review Queue.
The search project is an ongoing one. The time spent last year was essentially to change our search server to Solr. As of yet the benefits of this change aren’t apparent on the front-end. That’s the next portion of the search project. It’s always tough having a project that is long, complex, important, but with no visible improvement! But I’m very confident that we will reap huge benefits in the near future from the work last year.[/collapse][collapse title=”The “Maintenance Queue”” collapseid=”MaintenanceQueue”]Pretty much all the front-facing changes and updates you would have seen in the last year come from the maintenance queue. This is our queue of little fixes and changes. It’s anything which takes a half day to a couple days of a single developer. At any one time there are two devs allocated to the maintenance queue.[/collapse]

Scaling

On the scaling front, I checked in with our lead architect Pete who tells me we now have 8 physical machines with 68 cores and 432 GB of RAM , plus 17 virtual machines including 9 front-end servers, and we are serving up some 150 TB of traffic a month. In short, it’s a truckload of horsepower, and just scaling it all is a huge load of work. But it’s paid off, despite the enormous growth in traffic, you will notice that our sites are pretty snappy and rarely run into downtime. Security and fraud also take up quite a bit of time, and the bigger we get the more of an issue they become. Our level of response sophistication keeps rising as the threat level does. We are coping with both the scaling and security and fraud challenges, but it is a large amount of development work every month!

Read more on Envato Notes