Hello, this is my first post on #AltDevBlogADay so a quick introduction is in order. My name is Forrest Smith and I’m a software engineer at Uber Entertainment. Uber is a small studio so my work is all over the place. Hit detection, boss fights, steamworks, inventory, skills, weapons, etc. I’ve also been working on an iphone side project since roughly the dawn of time. One of my main goals in blogging is to share information that I’ve found useful in a concrete, easily applicable manner. We’ll see how that turns out.

Today I’m going to talk about the data for my iphone project and how I edited it. My project isn’t even a game, it’s a cocktail recipe book. It has over 1200 mostly unique recipes using almost 400 different ingredients. This is a lot of data to wrangle. There are already plenty of articles discussing how to store data – xml, json, binary, etc. However fewer articles discuss how to edit that data. Xml and json are both human readable, but not easy to edit at scale. A C# tool could be written from scratch to edit the data, but that seems heavy handed and wasteful for a side project.

A friend recommended using Google Spreadsheets as an editor and I could not be happier with the decision to use it. Typing data into a spreedsheet isn’t a particularly unique concept. What makes Google Spreadsheets cool is how trivially easy it is to write a C# tool to pull that data down. If you aren’t familiar with Google Documents service then I suggest you check it out. Any Google account, such as gmail, provides free access to basic MS Office type functionality. Documents (Word), Spreadsheets (Excel), Presentation (Powerpoint), etc. None of the Google offerings are as fully features or powerful as their retail MS counterparts, but they are all free and “in the cloud” for easy access by any internet enabled computer.

Google provides a .NET library (download) and dummy gmail account so anyone can download and run real code. It was compiled in Visual Studio 2010 with .NET 4. It works on my machine and I provide no further guarantees. :)

The demo project took me about 10 minutes to complete start to finish. I decided I would make a spreadsheet containing game developers and games and load it into a shiny tree view. Here’s what the data looks like.

Pretty simple and straight forward. A pair of column headers, the developer in column A, and a variable number of games to the right. What is the process to parse this data? Here’s a summary:

  1. Create a new spreadsheet service
  2. Set username and password
  3. Find the correct spreadsheet
  4. Find the correct worksheet
  5. Iterate over cells

And here is the actual code.

1
 
  2
 
  3
 
  4
 
  5
 
  6
 
  7
 
  8
 
  9
 
  10
 
  11
 
  12
 
  13
 
  14
 
  15
 
  16
 
  17
 
  18
 
  19
 
  20
 
  21
 
  22
 
  23
 
  24
 
  25
 
  26
 
  27
 
  28
 
  29
 
  30
 
  31
 
  32
 
  33
 
  34
 
  35
 
  36
 
  37
 
  38
 
  39
 
  40
 
  41
 
  42
 
  43
 
  44
 
  45
 
  46
 
  47
 
  48
 
  49
 
  50
 
  51
 
  52
 
  53
 
  54
 
  55
 
  56
 
  57
 
  58
 
  59
 
  60
 
  61
 
  62
 
  63
 
  64
 
  65
 
  66
 
  67
 
  68
 
  69
 
  70
 
  71
 
  72
 
  73
 
  74
 
  75
 
  76
 
  77
 
  78
 
  79
 
  80
 
  81
 
  82
 
  83
 
  84
 
  85
 
  
private void ImportFromGoogle()
 
  {
 
      // Login/Password to GMail Account
 
      string userName = "username_goes_here";
 
      string password = "password_goes_here";
 
   
 
      // Name of spreadsheet and worksheet to access
 
      string spreadsheetName = "Example Spreadsheet";
 
      string worksheetName = "Example Worksheet";
 
   
 
      // Connect to google's service with login/password
 
      SpreadsheetsService service = new SpreadsheetsService( spreadsheetName );
 
      service.setUserCredentials( userName, password );
 
   
 
      // Query spreadsheets
 
      SpreadsheetQuery spreadsheetQuery = new SpreadsheetQuery();
 
      SpreadsheetFeed spreadsheetFeed = service.Query(spreadsheetQuery);
 
   
 
      // Loop over all spreadsheets
 
      foreach( SpreadsheetEntry spreadsheet in spreadsheetFeed.Entries )
 
      {
 
          // Check if this the spreadsheet we want
 
          if( !spreadsheetName.Equals(spreadsheet.Title.Text, StringComparison.OrdinalIgnoreCase) )
 
              continue;
 
   
 
          // Query worksheets
 
          AtomLink worksheetLink = spreadsheet.Links.FindService(GDataSpreadsheetsNameTable.WorksheetRel, null);
 
          WorksheetQuery worksheetQuery = new WorksheetQuery(worksheetLink.HRef.ToString());
 
          WorksheetFeed worksheetFeed = service.Query(worksheetQuery);
 
   
 
          // Loop over worksheets
 
          foreach( WorksheetEntry worksheet in worksheetFeed.Entries )
 
          {
 
              // Check if this is the worksheet we want
 
              if( !worksheetName.Equals(worksheet.Title.Text, StringComparison.OrdinalIgnoreCase) )
 
                  continue;
 
   
 
              // Get cells
 
              AtomLink cellLink = worksheet.Links.FindService(GDataSpreadsheetsNameTable.CellRel, null);
 
              CellQuery cellQuery = new CellQuery(cellLink.HRef.ToString());
 
              CellFeed cellFeed = service.Query(cellQuery);
 
              AtomEntryCollection cellEntries = cellFeed.Entries;
 
   
 
              // Need to determine what column is what. Hardcoded for example.
 
              int devOffset = 0;
 
              int gameOffset = 1;
 
   
 
              // Loop over all entries finding the first entry on each row
 
              for( int i = 0; i < cellEntries.Count; ++i )
 
              {
 
                  // Continue if this entry is not the first entry of a row (and not the first row)
 
                  CellEntry baseEntry = cellEntries[i] as CellEntry;
 
                  if( baseEntry.Row == 1 || baseEntry.Column != 1 )
 
                      continue;
 
   
 
                  // Cell containing developer name
 
                  CellEntry devCell = cellEntries[i+devOffset] as CellEntry;
 
   
 
                  // Create a node for the tree view for this developer
 
                  TreeNode devNode = new TreeNode(devCell.Value);
 
   
 
                  // Loop over all games associated with this developer
 
                  int gameIndex = (i + gameOffset);
 
                  while( true )
 
                  {
 
                      // Get game cell, if it's on a new row we're done with this developer
 
                      CellEntry gameCell = cellEntries[gameIndex] as CellEntry;
 
                      if( gameCell.Row != devCell.Row )
 
                          break;
 
   
 
                      // Add game to developer tree node
 
                      devNode.Nodes.Add( new TreeNode(gameCell.Value) );
 
   
 
                      // Increment and validate index
 
                      ++gameIndex;
 
                      if( gameIndex >= cellEntries.Count )
 
                          break;
 
                  }
 
   
 
                  // Add developer node to list view
 
                  TV_Devs.Nodes.Add(devNode);
 
              }
 
          }
 
      }
 
  }

What do we get after all that? Something like this.

Tada! Spreadsheet data has successfully been transformed to C# data. This new and convenient data can be further transformed to our heart’s content.

A few things of note here. When you get the CellEntries object it contains a flat list of non-empty cells. This requires data to be properly formatted. It’s not a system I would recommend for AAA development, but for a solo project I’ve found it to be perfectly acceptable. If you want to do this in your own project then there are two special steps to take.

  • Include the three Google dlls – client, extensions, and spreadsheets. These can be found in my demo project or from the Google library.
  • Change your project target framework from “.Net Framework 4 Client Profile” to “.Net Framework 4″ as this enables System.Web framework access which is required to link the dlls.

Keeping with my focus on the real world, here is how I used this concept in my iphone project.  A standard recipe entry looks like this:

It defines a good bit of information including a variable number of recipe steps. Over 1200 of these recipes are loaded in addition to separate lists of glassware, drink types, measurement types, protips, ingredients, and lite edition recipes. After import the data is validated searching for typos, duplicates, and otherwise invalid entries. The data is then crunched converting most string values into integer ids. Finally, the data can be exported into whatever file format I desire to be loaded on the user platform. Here’s a picture of my tool to handle all this.

That’s what I did for my project and I hope at least somebody found this useful. How do you guys edit data for your side or AAA projects? Discuss in the comments below!