Archive for June, 2008

Search Is The Killer (Database) App

June 27th, 2008

Google proved that on the Web, search is the killer app. I think it’s a killer app for databases, too, and one that’s been egregiously absent from traditional database and spreadsheet products. In Oracle or Microsoft Access, for example, searching just doesn’t exist. A technical user can build a structured query, but they can’t search like they would at Google: type in Bob Johnson and instantly find all relevant data. If you think about what the Web would be like without search engines, you’ll realize what an amazing indictment of ordinary databases this represents.

Microsoft Excel, which is the world’s most widely-used de facto database, is hardly better. Search on Bob Johnson in Excel and you’ll be forced to hop from cell to cell, one at a time, among cells that contain Bob and Johnson right next to each other. If Bob is in the First Name column and Johnson is in the Last Name column – a pretty common way to organize data in a spreadsheet – you’re out of luck. Excel won’t find it.

TrackVia has always had a great search feature. It’s dirt simple, wicked fast, and smart. It will find all records in your data that are relevant to Bob Johnson, regardless of how those terms are structured in a data record (first name, last name, description, address, or some combination thereof). The results are cleanly displayed with yellow highlighting showing immediately why each record was a match.

Yet, we’ve had a cadre of customers who have been bugging us to make a good thing better. So this week we upgraded our search feature. It remains dirt simple and wicked fast, but it got smarter. While still automatically doing the right thing with common-sense requests, as when searching a product database for large red widget, it now has bells and whistles that make it more like the best Internet search engines. These include:

  • Phrases: Quotation marks, as in “New York” instead of New York, indicate that the terms must appear together.
  • Exact match: An equal sign, as in =Rob instead of Rob, will only return records with an exact match (i.e. Rob, but not Robert).
  • Negative match: A minus sign, as in white sox –red, means a term (i.e. red) must not appear in a record.
  • Dates: You can now search date and time fields with expressions like Oct 23, 2007 or simply February.
  • Empty fields: Searching on (none) will return records with blank fields.
  • Specific fields: Putting a field name before a term, as in first_name:Wilson, limits the search for that term to that one field.

You can also combine the above tricks to create powerful ad-hoc queries. For example, searching your food database for type=dessert apple -pie would return (presumably) apple cobbler, apple cake, and apple tarts. Here at TrackVia, we can compare the growth of our business this year versus last by doing two quick searches on our customer database – purchased:“June 2007” and purchased:“June 2008” – and comparing the number of results. Is it just me, or is that pretty neat?

Finally, you can now also explicitly search on the change notes, comments, and attachment names that are displayed in the History section of every TrackVia record. This means you can search the conversations and context around data records that represent business processes or workflow.

In my humble opinion, these upgrades make TrackVia the most searchable database ever built — and that’s a big deal. If you disagree, please let me know in the comments or directly by email!

Outsourcing Outages?

June 20th, 2008

The twelve-hour stretch of outages suffered by Google’s App Engine this week set off a debate in the blogosphere about whether “cloud computing” is a smart choice for Internet companies. My take is that beyond what’s already been discussed, two overlooked factors argue against cloud-based computing: the specificity of needs for Internet companies, and the importance of transparency during problems.

By way of background, cloud-based computing and storage services provide servers (often shared or “virtual” servers) and storage capacity on top of which an Internet company can build their application. So, for example, if TrackVia used a cloud-based infrastructure [we don’t – we control our own dedicated server farm at a Tier IV datacenter], then a TrackVia user who downloaded a file from TrackVia would cause our website, behind the scenes, to fetch that file from the cloud-based storage service instead of from a local server. Similarly, if a complex TrackVia report were to be executed, our website would have that report run on and delivered by a could-based virtual server, instead of a local machine. Cloud-based infrastructure can make life simpler for young Internet companies, but it also makes them susceptible to the cloud’s outages. In the last few months, there have been significant cloud-service problems at Amazon, HP, and others.

The cloud/outage debate is usually couched in terms of whether a company like TrackVia thinks they know more about keeping servers up and running than Google, Amazon, or the other mega players. Our uptime record is extremely good, but I’m not going to claim we know more about server management than Google does.

What we do have that Google doesn’t, though, is knowledge of our own needs. When planning infrastructure growth over time, TrackVia and all other web-based software companies have to balance cost, performance, and reliability, with an improvement in one often coming at the expense of the others. The right balance depends on the nature of the application and the customers that use it – in other words, there is a specificity of needs. A cloud provider optimizes that three-sided triangle in light of the aggregate needs of all of its clients. We optimize it in light of the specific needs of our application and our customers. We control our own infrastructure because we want to make our own choices about where to invest in any given month – Faster storage devices? Redundant load balancers? More database servers? More human beings? – instead of having the cloud make those decisions for us.

The second overlooked factor is transparency during problems. If we see an ominous trend in a certain server’s load during the day, we have very granular visibility into the problem and control over the possible solutions. We can communicate to customers what to expect. We can analyze the likelihood that it will happen again. We never have to say to a customer, “TrackVia is performing slowly because Google’s App Engine is performing slowly; we don’t know why; we don’t know how long it will last; we don’t know if it will happen again tomorrow.” Owning our infrastructure allows us to understand and communicate the why, how long, and will-it-happen-again issues. Scoring very well on those dimensions is as important as the uptime record itself. If you don’t believe this, ask your customers.

As cloud computing matures, I expect it to become an increasingly responsible choice for younger companies. But I hope the above illuminates why TrackVia at least decided to go our own way.

Pie charts a la mode

June 19th, 2008

Here at TrackVia, we’ve been hard at work cooking up new ways for you to view your database content. We’ve been keenly aware that sometimes you need more than just the traditional report grid view to better analyze your data. So, recently we’ve released the “statistics” view, which allows you to see your data in aggregate ways, including bar graphs and pie charts for your database fields if their underlying data types support them.

If you have a drop-down field in your database, for example, the statistics page will render a pie chart of what percentage of rows in your report have each option selected. If your field is a checkbox type, you will see a bar graph instead. In addition to charting, there are other neat things like totals, mean, median, and other statistical information on your numeric fields, as well as a report on most frequent values that appear in the report for other database columns.

And, just like a regular report, you can adjust the data sample being summarized by tuning the filters and display options. Give it a spin and let us know what you think!

Paperwork Purgatory: How TrackVia Saved My Marriage

June 17th, 2008

There is no part of my job I dislike more than filling out expense reimbursements. I once procrastinated for so long that the final tab was $38,000. I had to beg our company’s CFO to cut the check. Luckily, I’d kept the two years’ worth of receipts in a drawer, so my documentation was flawless, and I got paid. But the uncertainty in the outcome caused a significant amount of domestic tension. That’s a true story.

This year I’ve been using TrackVia to help me organize and submit expenses. I didn’t expect TrackVia to be a big improvement, since I generally think of our product as a database, not a spreadsheet. Of course there’s a grey area between those two types of software, but I usually consider TrackVia to be more useful for lists than for financial calculations.

To my surprise and delight, however, the chore is much faster and more accurate using TrackVia. Thus, for the benefit of paperwork-loathing professionals everywhere, I’m going to share my technique blow-by-blow in this blog post. Here goes.

First, I have a database called (naturally) Matt’s Expenses. The fields are:
   Purchase date – a date field
   Description – a short answer field
   Amount – a number field
   Status – a drop-down field, with choices of “paid” and “not yet paid”
   When Paid – a paragraph box

Whenever I buy something for TrackVia, I click on my Expenses database, add a record, and fill out the description and the amount. I use TrackVia’s “today” link to auto-fill the purchase date, and I have the status drop-down set to default to “not yet paid”. Therefore, it’s pretty quick to enter expenses as they’re incurred, instead of keeping a pile of receipts on my desk, which occasionally get bumped into waste basket. That’s benefit #1.

Now, some of my expenses come in the form of spreadsheets. For example, a few of the third-party marketing tools we use are billed automatically to my credit card whenever we’ve racked up a certain amount of advertising charges. Every few days my credit card will get charged without my knowing it. But these are easy to put into TrackVia: I download the charges from the advertising tool’s account management page, into Excel. I then upload that spreadsheet into TrackVia, using our field mapping wizard to change the marketing tool’s field names (e.g. “Payment Date”) into my field names (“Purchase Date”). Boom, 10 expenses are loaded into my database with no typing. That’s benefit #2.

On my TrackVia database overview page, I’ve removed the default report and put in a custom filtered view called Unpaid Expenses. It’s in chronological order, so it’s easy for me to keep an eye on how long it’s been since I’ve done expenses – I just look at the first date shown in that report. And it’s easy to see how much I’m owed at any given time — I click on “statistics” to see the total Amount for those unpaid expenses. Benefits #3 and #4.

When the list gets long enough that I decide it’s worth submitting a reimbursement form, TrackVia helps me de-dupe the list and make sure I’m not double-submitting expenses. For example, I might accidentally enter an expense for a book both on the day I order it and on the day it arrives. Before I submit the expense form, I use TrackVia’s quick search capability to pull up all expenses from Amazon. The search results immediately show me that my book is in there twice: from two weeks ago, with a status of paid, and from yesterday, with a status of unpaid. Two clicks later I’ve deleted the duplicate. Benefit #5.

TrackVia also helps me find gaps in regular expenses. I quick-search on “cell phone”, scroll to the bottom, glance at the Purchase Date and Status columns, and quickly see that Aprils’s bill is in there with a status of paid, and June’s bill is in there with a status of not yet paid, but May’s bill is missing. So I click “copy this record” on June’s expense record, change it to May, save it, and I just saved myself a $60 omission. Benefit #6.

But wait, there’s more! We have a standard company expense form template built in Excel, into which we’re supposed to type all of our expenses, then print it, sign it, and turn it in. My default report in TrackVia is laid out in the same column order as our reimbursement form: Date, Amount, Description. So I just click “Excel” in TrackVia, and when the spreadsheet opens, I select and copy the rows and paste them into the expense form (for Excel aficionados, I actually do Edit > Paste Special > Values). I save it, print it, and I’m done. No manual filling out the form. Benefit #7.

Finally, TrackVia gives me a great historical record (and here we get to the saving-my-marriage part). When the checks arrive from on high, I go back into the database and select the checkbox at the top of the Unpaid Expenses report. That selects all rows, and I use the “Continue” button and the short wizard to mark all of the expenses to a status of paid, en masse, and to enter the check number, the date I received it, and any other notes into the When Paid paragraph box. The expenses immediately disappear from my Unpaid Expenses report, showing a clean slate, ready to start the cycle anew. The benefit, though – our 8th and final benefit – is that when my wife, paying our bills some evening and still sore about the $38,000 near-fiasco, exclaims “Holy cow, there’s a charge for [some large amount] on our credit card! Did you remember to expense that??” I can do a Trackvia quick search and reply, “Yes, dear – it was part of the check I brought home for [some even larger amount] on [such-and-such date]. Would you like to know the check number?” My wife then looks at me admiringly, and perhaps makes kissing noises, because she’s so impressed that I’m on top of my game. And my marriage is saved.

(Okay, I took some poetic license in that last bit, but it’s a dry topic, so perhaps you’ll forgive me, gentle reader, for spicing it up a bit.)

Joking aside, it’s a good scheme, and it’s one of many unexpected ways in which TrackVia saves me time and keeps me organized. Now you know my scheme.

How about you? If you have novel ways of using TrackVia, please let us know.