Aaronontheweb

Hacking .NET and Startups

How to Make Any Operation Asynchronous in .NET

I’m in the middle of writing some updates to Quick and Dirty Feed Parser for use in a new personal project of mine; namely, I need to make QD Feed Parser work asynchronously so I can use it in conjunction with asynchronous controllers in ASP.NET MVC3, and eventually Silverlight + Windows Phone 7.

Asynchronous network IO is a breeze in every version of .NET since 1.1 – WebRequest and many others have supported asynchronous methods for as long as I’ve been using them. However, asynchronous disk IO, which I need to support in QD Feed Parser, is not something that comes so easily in .NET 4.0 and below.

StreamReader is the most user-friendly tool for reading files off of local disk because you don’t have to manage any buffers or do any of the sort of accounting that a class like FileStream requires1. Here’s how nice and easy life is using StreamReader, using actual code from QD Feed Parser as an example:

protected static FeedTuple DownloadXmlFromUri(Uri feeduri)
{
    string xmlContent;
    using (var reader = new StreamReader(feeduri.LocalPath))
    {
        xmlContent = reader.ReadToEnd();
    }

    return new FeedTuple{FeedContent = xmlContent, FeedUri = feeduri};
}

Given that StreamReader doesn’t have any built-in methods for making this asynchronous, what’s the best way to do this, given that the  async and await keywords coming down the pipe in .NET 5.0 aren’t fully available yet? The answer is: we’re going to wrap our disc IO operations using delegates.

Out of the box delegates have both a BeginInvoke and EndInvoke methods, which behave like any other standard asynchronous method call and you, the developer get to decide what functions are invoked on the background thread.

Here are the steps:

1. Create a delegate and a static worker method

protected delegate FeedTuple BackGroundWorker(Uri feeduri);

protected static FeedTuple DownloadXmlFromUri(Uri feeduri)
{
    string xmlContent;
    using (var reader = new StreamReader(feeduri.LocalPath))
    {
        xmlContent = reader.ReadToEnd();
    }

    return new FeedTuple { FeedContent = xmlContent, FeedUri = feeduri };
}

The delegate and the static worker method need to have the same signatures.

2. Define your own Begin[Operation] and End[Operation] methods

For Quick and Dirty Feed Parser, here is what I created, minus some integrity checks in order to make the code more readable:

public override IAsyncResult BeginDownloadXml(Uri feeduri, AsyncCallback callback)
{
    return FeedWorkerDelegate.BeginInvoke(feeduri, callback, 
		new FeedTuple());

}

public override FeedTuple EndDownloadXml(IAsyncResult asyncResult)
{
    var result = FeedWorkerDelegate.EndInvoke(asyncResult);
    return result;
}

Your begin method should return the IAsyncResult from your delegate – this way your caller can pass the asynchronous result back to the End[Operation] method and return your result. You can also use the IAsyncResult’s WaitHandle to put a block a calling thread if you wanted.

When dealing with delegates you have to be careful with your accounting – for every BeginInvoke there should be exactly one EndInvoke.

3. Calling the asynchronous methods

Here’s a quick unit test I wrote for testing this method in Quick and Dirty Feed Parser.

[Test, TestCaseSource("TestCases")]
public void CanDownloadXmlStreamAsync(string rsslocation)
{
    var feeduri = new Uri(rsslocation);
    var result = Factory.BeginDownloadXml(feeduri, null);
    var resultantTuple = Factory.EndDownloadXml(result);

    Assert.IsNotNull(resultantTuple);
    Assert.That(resultantTuple.FeedContent != String.Empty);
    Assert.AreEqual(feeduri, resultantTuple.FeedUri);
}

You should note that I’m effectively blocking the calling thread until the asynchronous worker thread returns a result in this method, and that’s because it’s easier to write unit tests this way.

The best practice for taking advantage of asynchronous method calls is to utilize a callback function, which you can do like so:

public void DownloadXmlStreamAsync(string rsslocation)
{
    var feeduri = new Uri(rsslocation);
    var result = Factory.BeginDownloadXml(feeduri, async => 
	{
		var callbackTuple = Factory.EndDownloadXml(async);
		Dispatcher.BeginInvoke(() => { 
		//... Do some client-side stuff with the data...		
		});
	});
}

And that’s it!


1It should be noted that FileStream does support asynchronous operations out of the box, but they’re a pain in the ass to use due to the reasons I just described.

If you enjoyed this post, make sure you subscribe to my RSS feed!



Quick and Dirty Feed Parser 0.3 Now Available

December 25, 2010 05:33 by Aaronontheweb in QDFeedParser // Tags: , // Comments (0)

My Christmas gift to the developer community: Quick and Dirty Feed Parser 0.3 is now available on CodePlex!

The big feature added in this release is the ability to perform IFeedFactory.CreateFeed operations asynchronously both for disc IO and network IO. I added this to QD Feed Parser primarily to support the use of asynchronous controllers in ASP.NET MVC3, and to support Silverlight and Windows Phone 7 eventually.

All classes which implement the IFeedFactory interface now have some new members:

IAsyncResult BeginCreateFeed(Uri feeduri, AsyncCallback callback);

IFeed EndCreateFeed(IAsyncResult asyncResult);

IAsyncResult BeginDownloadXml(Uri feeduri, AsyncCallback callback);

FeedTuple EndDownloadXml(IAsyncResult asyncResult);

If you’re at all familiar with the Begin[Procedure] End[Procedure] asynchronous programming pattern in .NET, then the purpose of these functions should be self-evident. Internally, BeginCreateFeed and EndCreateFeed are dependent upon the output of BeginDownloadXml and EndDownloadXml, but should a developer want to be able to work strictly with just the XML content itself then that’s why the [Begin/End]DownloadXml methods are publicly exposed.

Here’s a quick example which shows how you can use these functions to quickly and asynchronously parse RSS and Atom feeds:

Uri feeduri;
IFeedFactory factory;
if (UseFileSys)
{
    feeduri = new Uri(Path.GetFullPath(strFeedUri));
    factory = new FileSystemFeedFactory();
}
else
{
    feeduri = new Uri(strFeedUri);
    factory = new HttpFeedFactory();
}

factory.BeginCreateFeed(feeduri, async =>
{
    var feed = factory.EndCreateFeed(async);
    foreach (var feedItem in feed.Items)
    {
        Console.WriteLine("{0} {1}",
			feedItem.DatePublished, feedItem.Title);
        Console.WriteLine(feedItem.Link);
    }
});

If you’d like to play with this example of using Quick and Dirty Feed Parser asynchronously you can download the source from Github or the full release from CodePlex.

Originally I developed Quick and Dirty Feed Parser to support legacy NET 2.0 projects that wanted functionality that you could find using the System.ServiceModel.Syndication namespace. I’m developing it going forward because I think it’s easier to use than the ServiceModel classes and I’m using it on my own projects going forward.

If you’re interested in contributing to the project, we’d love to have you. Check out our issues list or come up with your own feature ideas that aren’t in it yet!

If you enjoyed this post, make sure you subscribe to my RSS feed!



Announcing the Release of Quick and Dirty Feed Parser

June 30, 2010 18:47 by Aaronontheweb in .NET, QDFeedParser // Tags: , , , // Comments (0)

Alternate headline: "never see XML again."

Ok, that may be a bit of a stretch. Regardless, I'm quite pleased to announce the launch of Quick and Dirty Feed Parser, a library for people who want a seamless way to use Atom and RSS feeds in their .NET 2.0+ applications without having to deal with the XML.

"Does the world really need another RSS/Atom parser," you ask? I'll let my source code speak for itself:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using QDFeedParser;

namespace QDFeedParserCMD
{
    class Program
    {
        private static readonly string DefaultFeedUri = "http://www.aaronstannard.com/syndication.axd";

        static void Main(string[] args)
        {
            try
            {
                string strFeedUri = args.Length == 0 ? DefaultFeedUri 
                    : args[0];

                Uri feeduri = new Uri(strFeedUri);
                IFeedFactory factory = new HttpFeedFactory();
                IFeed feed = factory.CreateFeed(feeduri);

                foreach(var feedItem in feed.Items)
                {
                    Console.WriteLine("{0} {1}", feedItem.DatePublished,
                        feedItem.Title);
                    Console.WriteLine(feedItem.Link);
                }
                //Just to prevent the window from instantly bailing out.
                Console.ReadKey(); 
            }
            catch(Exception ex)
            {
                Console.WriteLine("Exception: {0}", ex);
            }

        }
    }
}

That's about as complicated as it gets, folks. And feel free to download this QDFeedParser example from CodePlex

I wrote Quick and Dirty Feed Parser largely because I wanted to incorporate a really straightforward RSS/Atom parsing tool into a number of BlogEngine.NET modifications I have in mind in the near future, and I wanted a standard interface for dealing with RSS/Atom feeds that was available to me in .NET 2.0.

Quick and Dirty Feed Parser is my first OSS project so I would love your feedback, contributions, and most awesomely - examples of it actually being used! Check out the QDFeedParser project on CodePlex if you want to get involved!

QD Feed Parser in Action - The Great Wall of Geeks

No project worth it's salt would be complete without some sort of frivolous example, right? Well that's exactly why I created The Great Wall of Geeks.

Feel like your geeky blog belongs on the wall? Leave a comment below!

If you enjoyed this post, make sure you subscribe to my RSS feed!



How to Query a User's del.icio.us Feed with RestSharp

June 14, 2010 12:26 by Aaronontheweb in .NET, Del.icio.us // Tags: , , , , , // Comments (2)

I've been meaning to give RestSharp a go since I first started using Hammock in my startup project's codebase, just because I had heard some good things about RestSharp's auto-parsing capabilities.

This weekend I cobbled together a small example using del.icio.us' RSS feeds (not to be confused with its draconian REST API) for users and RestSharp performed magnificently, although its POCO -> XML Element mapping process requires a lot of experimentation before you get it just right.

Here's an example RSS feed for my personal del.icio.us account:

<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:wfw="http://wellformedweb.org/CommentAPI/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://web.resource.org/cc/">
  <channel>
    <title>Delicious/Aaronontheweb</title>
    <link>http://delicious.com/Aaronontheweb</link>
    <description>bookmarks posted by Aaronontheweb</description>
    <atom:link rel="self" type="application/rss+xml" href="http://feeds.delicious.com/v2/rss/Aaronontheweb?count=2"/>
    <item>
      <title>Windows Presentation Foundation (WPF) Dialog Boxes Overview</title>
      <pubDate>Fri, 04 Jun 2010 23:48:12 +0000</pubDate>
      <guid isPermaLink="false">[REMOVED]</guid>
      <link>http://msdn.microsoft.com/en-us/library/aa969773.aspx</link>
      <dc:creator><![CDATA[Aaronontheweb]]></dc:creator>
      <comments>[REMOVED]</comments>
      <wfw:commentRss>[REMOVED]</wfw:commentRss>
      <source url="http://feeds.delicious.com/v2/rss/Aaronontheweb">Aaronontheweb's bookmarks</source>
      <description>How to use dialog boxes in WPF (for WPF noobs like myself.)</description>
      <category domain="http://delicious.com/Aaronontheweb/">.net</category>
      <category domain="http://delicious.com/Aaronontheweb/">wpf</category>
      <category domain="http://delicious.com/Aaronontheweb/">dialogs</category>
      <category domain="http://delicious.com/Aaronontheweb/">tutorial</category>
      <category domain="http://delicious.com/Aaronontheweb/">nullable</category>
      <category domain="http://delicious.com/Aaronontheweb/">c#</category>
    </item>
    <item>
      <title>Developer's Guide: Data API Protocol - YouTube APIs and Tools - Google Code</title>
      <pubDate>Mon, 31 May 2010 22:43:47 +0000</pubDate>
      <guid isPermaLink="false">[REMOVED]</guid>
      <link>[REMOVED]</link>
      <dc:creator><![CDATA[Aaronontheweb]]></dc:creator>
      <comments>[REMOVED]</comments>
      <wfw:commentRss>[REMOVED]</wfw:commentRss>
      <source url="http://feeds.delicious.com/v2/rss/Aaronontheweb">Aaronontheweb's bookmarks</source>
      <description>Finally figured it out - how to sign all requests with my API key using a querystring. So simple, yet so difficult.</description>
      <category domain="http://delicious.com/Aaronontheweb/">youtube</category>
      <category domain="http://delicious.com/Aaronontheweb/">API</category>
      <category domain="http://delicious.com/Aaronontheweb/">key</category>
      <category domain="http://delicious.com/Aaronontheweb/">Google</category>
      <category domain="http://delicious.com/Aaronontheweb/">gdata</category>
    </item>
  </channel>
</rss>

The best way to utilize RestSharp to parse any sort of API response, whether it's custom REST XML, JSON, Atom, or RSS, is to first take a look at the raw response format and then to try to model a set of Data Transfer Objects (DTOs) which contain the response format elements you want to actually use in your application. Your DTOs just need to be simple POCO objects for this to work, but there are a set of POCO-XML Element matching rules you need to observe if you're using RestSharp's default deserializer - you can read them here.

Here are the classes that I modeled for this RSS format:

public class RssFeed
{
    public string Version { get; set; }
    public RssChannel Channel { get;set; }
}

public class RssChannel
{
    public string Title { get; set; }
    public string Description { get; set; }
    public string Link { get; set; }
    public RssItems Item { get; set; }
}

public class RssItems : List<item>{}

public class item
{
    public string Title { get; set; }
    public string Description { get; set; }
    public string Link { get; set; }
    public string Author { get; set; }
    public string Comments { get; set; }
    public string PubDate { get; set; }
}

This diagram explains how these DTO classes map to the del.icio.us RSS format:

del.icio.us restsharp POCO class to XML document mapping

ProTip: Handling Lists of XML Elements with POCO Classes

One thing that's a bit tricky is handling lists of XML elements, such as the <item> elements in this case. In order for RestSharp to deserialize them, you need create a List<T> object where the name of class T matches the name of the listed element, which is why my class name is item in this case.

Once you have your POCO classes in order, then you need to actually make requests against the del.icio.us feed for a particular user. Here's my code for doing that:

public class DeliciousRequest
{
    public const string DeliciousFeedBase = @"http://feeds.delicious.com/v2/rss/";

    private RestSharp.RestClient _client;

    public DeliciousRequest()
    {
        this._client = new RestClient
                          {
                              BaseUrl = DeliciousFeedBase
                          };
    }

    public RssFeed GetBookMarksForUser(string username)
    {
        var request = new RestRequest {RequestFormat = DataFormat.Xml, Resource = username};
        var response = this._client.Execute<RssFeed>(request);
        return response.Data;
    }

}

All RSS feeds for del.icio.us users can be found using http://feeds.delicious.com/v2/rss/{USERNAME}, therefore you can understand why the RestClient's BaseURL and the RestRequest's Resource arguments are defined as they are. Once you have your RestRequest defined, you simply call the client's Execute<T> method where T is the type of your POCO DTO class.

And that's it - RestSharp is easy, and I look forward to creating some more examples with it down the road.

More RestSharp Examples:

If you enjoyed this post, make sure you subscribe to my RSS feed!



Discussion: How to Use RestSharp / Hammock to Automatically Parse the YouTube Response Format into POCO Objects

June 8, 2010 17:35 by Aaronontheweb in .NET, YouTube // Tags: , , , , , // Comments (3)

If you've been following me on Twitter over the past couple of weeks, you might have noticed that I've been a little frustrated with the YouTube GData API lately. Simply put: XML makes me sad. Since that frustrated Tweet I've developed a solution using LINQ-to-XML and a bunch of hard-coded namespaces which isn't how I would prefer to do it.

I would much rather use the built in object deserialization capabilities in RestSharp or HammockREST. I'll be honest - I do not have a damn clue how to use Hammock's built-in deserialization capabilities. I tried tinkering with it on my own to no avail, and there's not much documentation to speak of.

RestSharp has some more detailed documentation on its deserialization capabilities, but it doesn't answer some lingering questions I have. So without further aideu, I'd like to solicit the opinion of the developer community.

<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:media="http://search.yahoo.com/mrss/" xmlns:openSearch="http://a9.com/-/spec/opensearch/1.1/" xmlns:gd="http://schemas.google.com/g/2005" xmlns:yt="http://gdata.youtube.com/schemas/2007" gd:etag="W/&quot;DkMMRn0zeSp7ImA9WxFWE0k.&quot;">
  <id>tag:youtube.com,2008:user:smartdraw:uploads</id>
  <updated>2010-05-31T22:21:27.381Z</updated>
  <category scheme="http://schemas.google.com/g/2005#kind" term="http://gdata.youtube.com/schemas/2007#video" />
  <title>Uploads by smartdraw</title>
  <logo>http://www.youtube.com/img/pic_youtubelogo_123x63.gif</logo>
  <link rel="related" type="application/atom+xml" href="http://gdata.youtube.com/feeds/api/users/smartdraw?v=2" />
  <link rel="alternate" type="text/html" href="http://www.youtube.com/profile_videos?user=smartdraw" />
  <link rel="http://schemas.google.com/g/2005#feed" type="application/atom+xml" href="http://gdata.youtube.com/feeds/api/users/smartdraw/uploads?v=2" />
  <link rel="http://schemas.google.com/g/2005#batch" type="application/atom+xml" href="http://gdata.youtube.com/feeds/api/users/smartdraw/uploads/batch?v=2" />
  <link rel="self" type="application/atom+xml" href="http://gdata.youtube.com/feeds/api/users/smartdraw/uploads?start-index=1&amp;max-results=25&amp;v=2" />
  <link rel="service" type="application/atomsvc+xml" href="http://gdata.youtube.com/feeds/api/users/smartdraw/uploads?alt=atom-service&amp;v=2" />
  <link rel="next" type="application/atom+xml" href="http://gdata.youtube.com/feeds/api/users/smartdraw/uploads?start-index=26&amp;max-results=25&amp;v=2" />
  <author>
    <name>smartdraw</name>
    <uri>http://gdata.youtube.com/feeds/api/users/smartdraw</uri>
  </author>
  <generator version="2.0" uri="http://gdata.youtube.com/">YouTube data API</generator>
  <openSearch:totalResults>30</openSearch:totalResults>
  <openSearch:startIndex>1</openSearch:startIndex>
  <openSearch:itemsPerPage>25</openSearch:itemsPerPage>
  <entry gd:etag="W/&quot;CEIHQn47eCp7ImA9WxFWE0k.&quot;">
    <id>tag:youtube.com,2008:video:NJPrllhYZrg</id>
    <published>2010-02-10T23:27:38.000Z</published>
    <updated>2010-05-31T21:48:53.000Z</updated>
    <category scheme="http://schemas.google.com/g/2005#kind" term="http://gdata.youtube.com/schemas/2007#video" />
    <category scheme="http://gdata.youtube.com/schemas/2007/categories.cat" term="People" label="People &amp; Blogs" />
    <category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="SmartDraw" />
    <category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="flowcharts" />
    <category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="visuals" />
    <category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="communicate" />
    <category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="visually" />
    <category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="communication" />
    <category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="powerpoint" />
    <category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="presentations" />
    <category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="mind" />
    <category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="maps" />
    <category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="software" />
    <category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="Business" />
    <category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="graphics" />
    <category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="strategic" />
    <category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="planning" />
    <category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="solutions" />
    <category scheme="http://gdata.youtube.com/schemas/2007/keywords.cat" term="tools" />
    <title>SmartDraw 2010 Guided Tour</title>
    <content type="application/x-shockwave-flash" src="http://www.youtube.com/v/NJPrllhYZrg?f=user_uploads&amp;d=AWaEdOkfU7AZas-hLyE9s8EO88HsQjpE1a8d1GxQnGDm&amp;app=youtube_gdata" />
    <link rel="alternate" type="text/html" href="http://www.youtube.com/watch?v=NJPrllhYZrg&amp;feature=youtube_gdata" />
    <link rel="http://gdata.youtube.com/schemas/2007#video.responses" type="application/atom+xml" href="http://gdata.youtube.com/feeds/api/videos/NJPrllhYZrg/responses?v=2" />
    <link rel="http://gdata.youtube.com/schemas/2007#video.ratings" type="application/atom+xml" href="http://gdata.youtube.com/feeds/api/videos/NJPrllhYZrg/ratings?v=2" />
    <link rel="http://gdata.youtube.com/schemas/2007#video.complaints" type="application/atom+xml" href="http://gdata.youtube.com/feeds/api/videos/NJPrllhYZrg/complaints?v=2" />
    <link rel="http://gdata.youtube.com/schemas/2007#video.related" type="application/atom+xml" href="http://gdata.youtube.com/feeds/api/videos/NJPrllhYZrg/related?v=2" />
    <link rel="http://gdata.youtube.com/schemas/2007#mobile" type="text/html" href="http://m.youtube.com/details?v=NJPrllhYZrg" />
    <link rel="self" type="application/atom+xml" href="http://gdata.youtube.com/feeds/api/users/smartdraw/uploads/NJPrllhYZrg?v=2" />
    <author>
      <name>smartdraw</name>
      <uri>http://gdata.youtube.com/feeds/api/users/smartdraw</uri>
    </author>
    <yt:accessControl action="comment" permission="allowed" />
    <yt:accessControl action="commentVote" permission="allowed" />
    <yt:accessControl action="videoRespond" permission="moderated" />
    <yt:accessControl action="rate" permission="allowed" />
    <yt:accessControl action="embed" permission="allowed" />
    <yt:accessControl action="syndicate" permission="allowed" />
    <gd:comments>
      <gd:feedLink href="http://gdata.youtube.com/feeds/api/videos/NJPrllhYZrg/comments?v=2" countHint="0" />
    </gd:comments>
    <media:group>
      <media:credit role="uploader" scheme="urn:youtube">smartdraw</media:credit>
      <media:description type="plain">Explains the features of SmartDraw, the software that allows you produce any visual, whether it's a flowchart or a floor plan, in a matter of minutes.
Download a free trial of SmartDraw here: http://www.smartdraw.com/downloads/?id=343742</media:description>
      <media:keywords>SmartDraw, flowcharts, visuals, communicate, visually, communication, powerpoint, presentations, mind, maps, software, Business, graphics, strategic, planning, solutions, tools</media:keywords>
      <media:player url="http://www.youtube.com/watch?v=NJPrllhYZrg&amp;feature=youtube_gdata" />
      <media:thumbnail url="http://i.ytimg.com/vi/NJPrllhYZrg/default.jpg" height="90" width="120" time="00:03:21" />
      <media:thumbnail url="http://i.ytimg.com/vi/NJPrllhYZrg/2.jpg" height="90" width="120" time="00:03:21" />
      <media:thumbnail url="http://i.ytimg.com/vi/NJPrllhYZrg/1.jpg" height="90" width="120" time="00:01:40.500" />
      <media:thumbnail url="http://i.ytimg.com/vi/NJPrllhYZrg/3.jpg" height="90" width="120" time="00:05:01.500" />
      <media:thumbnail url="http://i.ytimg.com/vi/NJPrllhYZrg/hqdefault.jpg" height="360" width="480" />
      <media:title type="plain">SmartDraw 2010 Guided Tour</media:title>
      <yt:duration seconds="402" />
      <yt:uploaded>2010-02-10T23:27:38.000Z</yt:uploaded>
      <yt:videoid>NJPrllhYZrg</yt:videoid>
    </media:group>
    <gd:rating average="4.4444447" max="5" min="1" numRaters="9" rel="http://schemas.google.com/g/2005#overall" />
    <yt:statistics favoriteCount="9" viewCount="16206" />
    <yt:rating numDislikes="1" numLikes="8" />
  </entry>
</feed>

Now, here are use cases for how I might want to use this format:

  1. The GData API will only serve a maximum of 50 entries at any given time - one way to paginate through all of the entries in one go is to parse the link rel="next" field and query that URL until the field no longer exists. Is there a way you can use a POCO class in Hammock or RestSharp to automatically grab this field between queries against the API?
  2. Imagine you create a POCO class which contains the YouTube video ID, the author's username, the number of comments on the video, the keywords for the video, the number of views, and the number of the number of favorites. How would you structure this class such that RestSharp or HammockREST can automatically parse it from an ATOM response format like the one above? Bear in mind that these fields come from four of the five different XML namespaces (atom, Media RSS [media], GData [gd], and YouTube [yt]) used in this response format.

If you enjoyed this post, make sure you subscribe to my RSS feed!



Frustration Central: Parsing the DateTime Values from the SlideShare REST API

June 2, 2010 09:49 by Aaronontheweb in .NET, SlideShare // Tags: , , , , // Comments (8)

I feel a little bad about posting this given that Jon Boutelle, the CTO of SlideShare, already admitted that this portion of the SlideShare 2.0 REST API sucks and that they're going to fix it eventually, but given that I'm in the middle of rewriting my original SlideShare Presentation XML deserializer I wanted to post my current solution to this problem and solicit the opinion of the .NET community in order to find a more elegant solution, if one exists.

Here's the problem - if you're trying to query any number of presentation objects from SlideShare, you're inevitably going to make a call to get_slideshow or to some method which depends on it. That's an unavoidable fact of life when you're dealing with the SlideShare API. For the most part, the SlideShare API's response format is intuitive and intelligible - here's an example response using actual data:

<?xml version="1.0" encoding="utf-8"?>
<Slideshow>
  <ID>3997766</ID>
  <Title>Startup Metrics 4 Pirates (Montreal, May 2010)</Title>
  <Description>slides from my talk at Startup Camp Montreal 6 (May, 2010)</Description>
  <Status>2</Status>
  <Username>dmc500hats</Username>
  <URL>[removed for fomatting reasons]</URL>
  <ThumbnailURL>[removed for fomatting reasons]</ThumbnailURL>
  <ThumbnailSmallURL>[removed for fomatting reasons]</ThumbnailSmallURL>
  <Embed>
    [removed for fomatting reasons]
  </Embed>
  <Created>Thu May 06 14:10:46 -0500 2010</Created>
  <Language>en</Language>
  <Format>ppt</Format>
  <Download>1</Download>
  <Tags>
    <Tag Count="1" Owner="1">startup</Tag>
    <Tag Count="1" Owner="1">scmtl</Tag>
    <Tag Count="1" Owner="1">leanstartup</Tag>
    <Tag Count="1" Owner="1">acquisition</Tag>
    <Tag Count="1" Owner="1">activation</Tag>
    <Tag Count="1" Owner="1">poutin</Tag>
    <Tag Count="1" Owner="1">pirate</Tag>
    <Tag Count="1" Owner="1">metrics</Tag>
    <Tag Count="1" Owner="1">referral</Tag>
    <Tag Count="1" Owner="1">aarrr</Tag>
    <Tag Count="1" Owner="1">retention</Tag>
    <Tag Count="1" Owner="1">revenue</Tag>
  </Tags>
  <NumDownloads>18</NumDownloads>
  <NumViews>1002</NumViews>
  <NumComments>0</NumComments>
  <NumFavorites>3</NumFavorites>
  <NumSlides>67</NumSlides>
   <RelatedSlideshows>
    <RelatedSlideshowID rank="6">89026</RelatedSlideshowID>
    <RelatedSlideshowID rank="4">602558</RelatedSlideshowID>
    <RelatedSlideshowID rank="3">629696</RelatedSlideshowID>
    <RelatedSlideshowID rank="5">629833</RelatedSlideshowID>
    <RelatedSlideshowID rank="9">1064559</RelatedSlideshowID>
    <RelatedSlideshowID rank="10">1566287</RelatedSlideshowID>
    <RelatedSlideshowID rank="1">2992302</RelatedSlideshowID>
    <RelatedSlideshowID rank="2">3017886</RelatedSlideshowID>
    <RelatedSlideshowID rank="8">3387416</RelatedSlideshowID>
    <RelatedSlideshowID rank="7">3951684</RelatedSlideshowID>
   </RelatedSlideshows>
  <PrivacyLevel>0</PrivacyLevel>
  <SecretURL>0</SecretURL>
  <AllowEmbed>0</AllowEmbed>
  <ShareWithContacts>0</ShareWithContacts>
</Slideshow>

Intuitive, no? However, let's take a closer look at the <Created> field, which contains the date that this SlideShare presentation was originally uploaded:

<Created>Thu May 06 14:10:46 -0500 2010</Created>

What, pray tell, is this? It appears that SlideShare REST 2.0's date format consists of the following elements in left to right order:

<Created>Day of week name, Month name, Day of month, Hours:Minutes:Seconds, UTC offset, Year</Created>

Unfortunately, this format is not one of the many supported Standard Date and Time Formats in .NET nor is it a UNIX time format, so DateTime.Parse and the other built-in DateTime parsing variants in .NET will not be able to parse this DateTime data from the SlideShare API. Thus, I created my own admittedly ugly solution to this problem, which I will reveal below, every ugly piece by ugly piece.

Despite how ugly this solution is, it's been tested pretty thoroughly and works (as far as I know.) The only thing it does not do, currently, is use the UTC offset in any way, shape or form, which is a modification I am currently working on in my new version of the SlideShare presentation deserializer which uses these functions.

The first issue is that there's no numeric value for the month - you have to map a three-letter code to its correspodning month, i.e. "Mar" = 3, "May" = 5, and so forth. Here's the function I made for that very purpose:

public static int GetMonth(string slideShareDatetime)
{
    try
    {   //Skip past the three-letter code for the day of the week and the space in between.
        string MonthStr = slideShareDatetime.Substring(4, 3).ToLower();
        int returnValue = 1;
        switch (MonthStr)
        {
            case "jan":
                returnValue = 1;
                break;
            case "feb":
                returnValue = 2;
                break;
            case "mar":
                returnValue = 3;
                break;
            case "apr":
                returnValue = 4;
                break;
            case "may":
                returnValue = 5;
                break;
            case "jun":
                returnValue = 6;
                break;
            case "jul":
                returnValue = 7;
                break;
            case "aug":
                returnValue = 8;
                break;
            case "sep":
                returnValue = 9;
                break;
            case "oct":
                returnValue = 10;
                break;
            case "nov":
                returnValue = 11;
                break;
            case "dec":
                returnValue = 12;
                break;
            default:
                throw new InvalidOperationException("Unable to recognize month" + MonthStr + " in SlideShareDate.");
        }
        return returnValue;
    }
    catch (Exception ex)
    {
        logger.ErrorException("Unable to parse month from SlideShare presentation.", ex);
        throw;
    }
}

The next thing we have to do is extract the numeric day of the week, which I do using the function below:

public static int GetDay(string slideShareDatetime)
{
    string strRegexPattern = @"((\d|\d{2})\b){1}";
    Regex r = new Regex(strRegexPattern);
    string matchingDay = r.Match(slideShareDatetime).ToString();
    return Convert.ToInt32(matchingDay);
}

I tested several regular expressions numerous times before I settled on this one, which I liked for its simplicity - all it does is look for the first instance of a one or two numeric character word in a given string.

Next, we have to extract the year from the string, which I do using the function below:

public static int GetYear(string slideShareDatetime)
{
    string YearStr = slideShareDatetime.Substring(slideShareDatetime.Length - 4, 4);
    return Convert.ToInt32(YearStr);
}

All this function does is extract the last four characters from the string (the date, in this case) and converts it into an integer. I suppose this function will break sometime around the year 10,000, or whenever SlideShare adds some whitespace to the end of their XML response for the Created field, whichever comes first.

Lastly we have to extract the hour:minutes:seconds values into some sort of intelligible format, so thus I present you with yet another cringe-worthy function:

public static Dictionary GetTime(string slideShareDatetime)
{
    try
    {
        Dictionary returnValue = null;

        string strRegexPattern = @"((\d{2}:\d{2}:\d{2})\b)";
        Regex r = new Regex(strRegexPattern);

        string matchingDay = r.Match(slideShareDatetime).ToString();
        if (matchingDay == String.Empty)
            throw new InvalidOperationException("Unable to parse time of day from SlideShareDate:" + slideShareDatetime);

        string[] TimeParts = matchingDay.Split(':');
        if (TimeParts.Length < 3)
            throw new InvalidOperationException("Invalid date/time parsed from SlideShareDate:" + slideShareDatetime);

        returnValue = new Dictionary();
        returnValue.Add("hours", Convert.ToInt32(TimeParts[0]));
        returnValue.Add("minutes", Convert.ToInt32(TimeParts[1]));
        returnValue.Add("seconds", Convert.ToInt32(TimeParts[2]));

        return returnValue;
    }
    catch (Exception ex)
    {
        logger.ErrorException("Failed to parse time from presentation.", ex);
        throw;
    }
}

And then we put it all together in one, big, happy function:

public static DateTime ParseSlideShareDateTime(string slideShareDatetime)
{
    int Month = GetMonth(slideShareDatetime);
    int Year = GetYear(slideShareDatetime);
    int Day = GetDay(slideShareDatetime);
    Dictionary Time = GetTime(slideShareDatetime);
    DateTime properDate = new DateTime(Year, Month, Day, Time["hours"], Time["minutes"], Time["seconds"]);
    return properDate;
}

Yeah. So - can anybody think of a much more, ahem, "robust" way of parsing the Created field from the SlideShare API? I would love to hear it! :)

If you enjoyed this post, make sure you subscribe to my RSS feed!



Making Basic Requests Against the SlideShare API Using HammockREST

May 27, 2010 15:49 by Aaronontheweb in SlideShare // Tags: , , , , // Comments (4)

The project I'm currently working on involves numerous REST APIs from a multitude of very different services. In my initial prototype, which I've since scrapped, I went with trying to use a local wrapper for each REST API incorporated into my project, meaning I used the GData library for handling YouTube, FlickrNET for Flickr, etc...

As you can imagine, having N libraries for N different REST APIs in my service snowballed into a maintainability nightmare. In my new redesign I decided to use a generic REST wrapper in .NET, and as it so happens there are two full API-agnostic REST libraries in .NET: HammockREST and RestSharp. If there's one thing you can fault both of these libraries for, it's lack of documentation - they're both new comers to the .NET open source community, so I'm going to contribute some documentation for Hammock and RestSharp, starting with the SlideShare API for Hammock.

If you take a look at HammockREST's documentation, you can see how to structure a generic REST request for the Twitter API. Below is my full function for calling the get_slideshows_by_user from the SlideShare API using HammockREST; we'll start from the code and work backwards to explain what's going on underneath the hood.

public static string Query(string username)
{
    RestClient client = new RestClient
    {
        Authority = "http://www.slideshare.net/api/",
        VersionPath = "2"
    };
    client.AddHeader("User-Agent", "Hammock");

    RestRequest request = new RestRequest
    {
        Path = "get_slideshows_by_user"
    };

    string timestamp = QueryHelper.GetCurrentTimestamp();

    request.AddParameter("api_key", Constants.SlideShareAPIKey);
    request.AddParameter("ts", timestamp);
    request.AddParameter("hash", QueryHelper.CalculateSHA1(Constants.SlideShareSharedSecret + timestamp, Encoding.UTF8));
    request.AddParameter("username_for", username);

    RestResponse response = client.Request(request);

    return response.Content;
}

Bear in mind that this is just meant to be simple example code to illustrate how HammockREST works with the SlideShareAPI, not something you'd stick in a production environment.

If you're unfamiliar with the SlideShare API remember that every request, even unauthenticated ones, requires the following three parameters:

  1. "api_key" - your SlideShareAPI developer key;
  2. "ts" - a UNIX-style timestamp; and
  3. "hash" - a hash of your SlideShareAPI developer shared secret plus the UNIX-style timestamp using SHA1. The hash must be in the order of secret + timestamp.

Given that I'm lazy, I borrowed the timestamp and SHA1 function Frederik Vig's original SlideShareNET wrapper. I'll post the source for both of those at the bottom of this post.

With that out of the way, let's look at the different HammockREST components in this code snippet and understand how they work:

RestClient client = new RestClient
{
     Authority = "http://www.slideshare.net/api/",
     VersionPath = "2"
};

This code defines a new Hammock.RestClient object, which specifies the general target for all subsequent API requests. Recall that the base URI for making requests against the SlideShare API is, as of writing this, http://www.slideshare.net/api/2/. In this source code I've expressed that as a combination of two properties:

  • "Authority" - which is set to the root URI for all requests (http://www.slideshare.net/api/.)
  • "VersionPath" - which in this case is version #2.

Whenever I execute a Hammock.RestRequest object using this Hammock.RestClient instance, all of those Requests will be executed against this URI: http://www.slideshare.net/api/2/. I could have just as easily accomplished this with the following code:

RestClient client = new RestClient
{
     Authority = "http://www.slideshare.net/api/2/"
};

The point is that the VersionPath property is optional - you can just specify the full base URL in the Authority property if you'd prefer to do it that way.

Let's take a look at the Hammock.RestClient instance:

RestRequest request = new RestRequest
{
     Path = "get_slideshows_by_user"
};

The way Hammock (and RestSharp, as you'll see) construct the URLs for REST requests is the following formula:

Request URL = {authority[/versionpath]}/{path}

Each individual RestRequest object is meant to map to a specific REST method call, so in this instance I'm trying to call the get_slideshows_by_user method from the SlideShare API. The code I've shown you thus far accomplishes that effectively, and in sticking with this API/method metaphor, we need to supply our REST API method with some arguments:

request.AddParameter("api_key", Constants.SlideShareAPIKey);
request.AddParameter("ts", timestamp);
request.AddParameter("hash", QueryHelper.CalculateSHA1(Constants.SlideShareSharedSecret + timestamp, Encoding.UTF8));
request.AddParameter("username_for", username);

This code is straight-forward enough - I add my three required parameters for any SlideShare request, and then I added parameter required specifically for get_slideshows_by_user, username_for, which specifies the SlideShare user whose presentations we want to retrieve via this API call.

Finally, we use the client to actually complete the REST API request:

RestResponse response = client.Request(request);

This returns a Hammock.RestResponse object, which is a container that includes data about the status of the request (i.e. if you ran into a HTTP error, other errors, etc...) and the API's response in string format.

That's the long and the short of it - in subsequent examples I'll show you how to automatically deserialize XML content into business objects using HammockREST and I'll show you how to do all of this using RestSharp too.

As promised, here's the functions (originally developed by Frederik Vig) which I used for generating the timestamps and hashes:

public static string GetCurrentTimestamp()
{
    return Convert.ToInt32(Math.Floor((DateTime.UtcNow - new DateTime(1970, 1, 1, 0, 0, 0)).TotalSeconds)).ToString();
}
using System.Security.Cryptography;

public static string CalculateSHA1(string text, Encoding enc)
{
    byte[] buffer = enc.GetBytes(text);
    var cryptoTransformSHA1 = new SHA1CryptoServiceProvider();

    return BitConverter.ToString(cryptoTransformSHA1.ComputeHash(buffer)).Replace("-", "").ToLower();
}

If you enjoyed this post, make sure you subscribe to my RSS feed!



Search

About

My name is Aaron, I'm an entrepreneur and a .NET developer who develops web, cloud, and mobile applications.

I left Microsoft recently to start my own company, MarkedUp - we provide analytics for desktop developers, focusing initially on Windows 8 developers.

You can find me on Twitter or on Github!

Recent Comments

Comment RSS

Sign in