iWitness, our project to create a tool to aggregate social media by time and place, is now well underway, and we’re looking forward to sharing some of our thinking about the design of the tool with you in the coming weeks.
In the meantime, we’ve asked our development partners at EdgeCase to provide their perspective on the technical side of the process. Here’s what Mike Doel, project manager at EdgeCase, had to say:
the Process
At EdgeCase, we use an agile development process to rapidly produce functionality with a minimum of waste. We combine elements of “Extreme Programming” and “Behavior Driven Development” for a custom process founded on stakeholder collaboration that works.
The project began with a series of exercises designed to ensure we started the project well. Over the course of three days, we agreed to domain language, identified key concepts for the project, drew wireframes, and pulled together a set of features that were essential to verifying the intended direction of the project. At the end, the room was filled with sticky notes and flip charts, and everyone involved had gained a better understanding of what work would be required over the next several weeks.
Once the feature set was agreed to, the next step was to capture all of those requirements into a discrete set of user stories. Each one represents some distinct requirement that can be understood and developed in isolation. Unlike more traditional waterfall models, the agile methods we follow utilize short bursts of work that allow for rapid feedback and evaluation.
The user story is one of the basic building blocks of this process. We use a tool from Pivotal Labs called Pivotal Tracker to manage these. Each story includes a title, a description of the feature, and an estimate of the effort involved. As people work on the story, they have the opportunity to add comments and ask questions.
We develop in 2-week cycles called iterations. Each iteration starts with an Iteration Planning Meeting (IPM) where we discuss what features we’re going to build for the next two weeks. At the end of the cycle, we review the stories developed during the iteration and get them signed off by the client. We also perform a retrospective where we reflect back on the prior two weeks and identify what things we’re doing that are helping the project and which parts of our process could be improved.
The daily routine in an agile product is fairly simple. Every day, we hold a brief 5- to 10-minute standup meeting, where we literally stand up the whole time, which helps keep the meetings short. In it, we cover the progress for the day and discuss any issues the team is having. The rest of the day is spent pair programming.
When developers work together in a paired fashion, the total is greater than the sum of the parts. The benefits include:
- Every line of code is subjected to an instant code review by someone who is familiar with the code.
- It prevents the formation of silos of knowledge.
- The pressure of having a peer actively inspecting and participating in your work forces you to focus and avoid distractions.
- If one developer has trouble solving some problem, there’s another person to help him or her get unstuck quickly.
That’s a high-level overview of some elements of the agile process we at EdgeCase use every day in the development of iWitness. Now, let’s turn our attention to some of the cool technologies we’re using to build the product itself.
the Technologies
One of the key design constraints we face with iWitness is that it needs to be capable of standing on its own without requiring sophisticated IT skills to manage. Most newsrooms don’t have people with those abilities. Also, the principal funding for the work being done on iWitness comes from a grant from the John S. and James L. Knight Foundation. With limitations in funding and support staff, along with conditions imposed by the grant (e.g., the final product must be released as open source), we concluded that the proper architecture for iWitness is as a wholly client-side application.
The primary feature for iWitness is to find social media activity for a specific time and place. For example, a journalist writing a story about an Occupy movement protest may want to find tweets from a certain city block at a given time. For each social media service supported by iWitness, we need to query the service via an exposed API endpoint and synthesize the results in an intuitive manner. As this is being done in the browser, that implies that the code is being written in javascript — the lingua franca for dynamic web applications.
Recently, a slew of javascript frameworks have been released to help application developers provide structure to their client-side applications. Some of the more popular ones are ember.js, backbone.js, and knockout.js. We have decided to base iWitness on ember because it offered more structure and better suited our needs.
Ember is an example of an MVC (Model, View, Controller) framework. Models encompass primary domain objects like search criteria and a result from Twitter. The view layer is home to the templates for how pages look when rendered — things like the order of form fields, the text on the page, and the hierarchy of elements that the user sees. A controller is the glue between the model and the view.
Although we will eventually deploy iWitness as a standalone client application, there are still server-side tools that can make this easier. For example, we get performance benefits if we deliver the iWitness javascript assets as a single file. While developing the code, however, it’s better for the javascript to be broken into logical chunks that can be understood and tested in isolation.
We’re using an asset packaging and dependency management tool called Sprockets to bridge the gap. With Sprockets, our developers can create the code in small chunks and then bundle it together for efficient delivery. Other tools and packages we’re using for similar reasons include a simple Ruby-based web server named rackup, the uglifier gem to compress javascript, and the LESS dynamic stylesheet language.
Working on iWitness to this point has been a ton of fun. The partnership between EdgeCase and Adaptive Path has been great for both companies, and we think that will show in the work we produce together. Keep your eyes on this space for further announcements about the release of iWitness for general use and the call for open-source developers to join us in making it even better.