Web API and API Data Retreiver

Summary

In this blog post I’m going to show how to create a Web API in Visual Studio.  Then I’m going to show how to setup IIS 7 to run the API on your PC (or a server).  I’m also going to create a retriever and show how to connect to the Web API and read data.  I’ll be using JSON instead of XML so I’ll also show what tricks you’ll need to know in order to implement your interface correctly.  Finally, I’ll demonstrate how to troubleshoot your API using fiddler.


This is a very long post.  I had toyed with the idea of breaking this into multiple parts, but this subject turned out to be too difficult to break-up in a clean manner.  If you are having trouble getting this to work properly or you think I missed something leave a message in the comments and I’ll correct or add to this article to make it more robust.

Web API Retriever

I’m going to build the retriever first and show how this can be tested without an API.  Then I’ll cover the API and how to incorporate IIS 7 into the whole process.  I’ll be designing the API to use the POST method.  The reason I want to use a POST instead of GET is that I want to be able to pass a lot of variables to request information from the API.  My demo will be a simulation of a chain of stores that consolidate their inventory data into a central location.  Headquarters or an on-line store application (i.e. website) can send a request to this API to find out what inventory a particular store has on hand.  

The retriever will be a simple console application that will use an object to represent the request data.  This object will be serialized into a JSON packet of information posted to the API.  The request object will look like this:

public class ApiRequest
{
  public int StoreId { get; set; }
  public List<int> ProductId { get; set; }
}


This same object will be used in the API to de-serialize the JSON data.  We can put the store id in this packet as well as a list of product ids.  The data received back from the API will be a list of inventory records using the following two objects:

public class InventoryRecord
{
  public int ProductId { get; set; }
  public string Name { get; set; }
  public int Quantity { get; set; }
}


public class ApiResponse
{
  public List<InventoryRecord> Records = new  

         List<InventoryRecord>();
}

As you can see, we will receive one record per product.  Each record will contain the product id, the name and the quantity at that store.  I’m going to dummy out the data in the API to keep this whole project as simple as possible.  Keep in mind, that normally this information will be queried from a large database of inventory.  Here’s the entire retriever:

public class WebApiRetriever
{
  private readonly string apiURLLocation =  

             ConfigurationManager.AppSettings[“ApiURLLocation“];
  private readonly string apiAuthorization =  

          ConfigurationManager.AppSettings[“ApiCredential“];

  public void Retreiver()
  {
    var serializer = new JsonSerializer();

    var apiRequest = new ApiRequest
    {
      StoreId = 1,
      ProductId = { 2, 3, 4 }
    };

    var request = 
        (HttpWebRequest)WebRequest.Create(apiURLLocation + “”);
    request.ContentType = “application/json; charset=utf-8“;
    request.Accept = “application/json“;
    request.Method = “POST“;
    request.Headers.Add(HttpRequestHeader.Authorization, 

         apiAuthorization);
    request.UserAgent = “ApiRequest“;

    //Writes the ApiRequest Json object to request
    using (var streamWriter = new 
           StreamWriter(request.GetRequestStream()))
    {             

      streamWriter.Write(
           JsonConvert.SerializeObject(apiRequest));
      streamWriter.Flush();
    }

    var httpResponse = (HttpWebResponse)request.GetResponse();

    using (var streamreader = new  
           StreamReader(httpResponse.GetResponseStream()))
    using (var reader = new JsonTextReader(streamreader))
    {
      var storeInventory = 

          serializer.Deserialize<ApiResponse>(reader);
    }
  }
}


Some of the code shown is optional. I put the URL location and credentials into variables that are stored in the app.config file.  You can add this to your app.config file:


<appSettings>
    <add key=”ApiURLLocationvalue=” 

       http://www.franksdomain.com/WebApiDemo/api/MyApi/“/>
    <add key=”ApiCredentialvalue=”ABCD“/>
</appSettings>

The URL will need to be changed to match the URL that you setup on your IIS server (later in this blog post).  For now you can setup a redirect in your “hosts” file to match the domain in the app setting shown above.

Navigate to C:WindowsSystem32driversetc and edit the “hosts” file with a text editor.  You’ll see some sample text showing the format of a URL.  Create a domain name on a new line like this:

127.0.0.1        www.franksdomain.com

You can make up your own URL and you can use a URL that is real (technically, franksdomain.com is a real URL and it’s not mine).  If you use a real URL your computer will no longer be able to access that URL on the internet, it will redirect that URL to your IIS server (so be aware of this problem and try to avoid using real URLs).  The IP address 127.0.0.1 is a pointer to your local machine.  So we’re telling your browser to override www.franksdomain.com and redirect the request to the local machine.

Now you should be able to test up to the request.GetResponse() line of code.  That’s where the retriever will bomb.  Before we do this, we need to download and install Fiddler (assuming you don’t already have fiddler installed).  Click here to download fiddler and install it.  Now start up fiddler and you’ll see something like this:


Now run the retriever application until it bombs.  Fiddler will have one line in the left pane that is in red.  Click on it.  In the right pane, click on the “Inspectors” tab and then click on “JSON” sub-tab.  You should see something like this:


In the right side top pane, you’ll see your JSON data.  If your data is shown as a tree-view control then it is formatted correctly as JSON and not just text.  If you serialized your object incorrectly, you will normally see an empty box.  Notice that the store is set to “1” and there are three product ids being requested.

The Web API

The web API will be an MVC 4 application with one ApiController.  The API controller will use a POST method. 

So let’s start with a class that defines what information can be posted to this API.  This is the exact same class used in the retriever:

public class ApiRequest
{
  public int StoreId { get; set; }
  public List<int> ProductId { get; set; }
}


Make sure this class is not decorated with the [Serializable] attribute.  We’re going to use a [FromBody] attribute on the API and the object variables will not bind if this object is setup as serializable (I discovered this fact the hard way).  As you can see by the list definition we can pass a long list of product ids for one store at a time.  We expect to receive a list of inventory back from the API.


The response back to the calling application will be a list of inventory records containing the product id (which will be the same number we passed in the request), the name of the product and the quantity.  These are also the same objects used in the retriever:


public class InventoryRecord
{
  public int ProductId { get; set; }
  public string Name { get; set; }
  public int Quantity { get; set; }
}


public class ApiResponse
{
  public List<InventoryRecord> Records = new  

         List<InventoryRecord>();
}



The entire API controller in the MVC application looks like this:


public class MyApiController : ApiController
{
  [HttpPost]
  [ActionName(“GetInventory“)]
  public HttpResponseMessage GetInventory([FromBody] ApiRequest request)
  {
    if (request == null)
    {
      return Request.CreateErrorResponse(HttpStatusCode.BadRequest, 

             “Request was null“);
    }
       
    // check authentication
    var auth = ControllerContext.Request.Headers.Authorization;
       
    // simple demonstration of user rights checking.
    if (auth.Scheme != “ABCD“)
    {
      return Request.CreateErrorResponse(HttpStatusCode.BadRequest, 

             “Invalid Credentials“);
    }
   
    ApiResponse apiResponse = new ApiResponse();

    // read data from a database
    apiResponse.Records = DummyDataRetriever.ReadData(request.ProductId);

    // convert the data into json
    var jsonData = JsonConvert.SerializeObject(apiResponse);

    var resp = new HttpResponseMessage();
    resp.Content = new StringContent(jsonData, Encoding.UTF8, 
                   “application/json“);
    return resp;
  }
}


The controller is a post method controller that looks for an ApiRequest JSON object in the body of the posted information.  The first thing we want to check for is a null request.  That seems to occur most often when a bot crawls a website and hits an API.  If we’re lucky, then bots will not find their way in, but I always code for the worse case situation.  The next part will check the header for the authorization.  I didn’t cover this in the retriever, but I stuffed a string of letters in the authorization variable of the header.  This was setup to be “ABCD” but in a real application you’ll need to perform a database call to a table containing GUIDs.  These GUIDs can be assigned to another party to gain access to your API.  In this example the shopping website application will have its own GUID and each store will have a GUID that can be setup to restrict what each retriever can access.  For instance, the website GUID might have full access to every store to lookup information using this API, but a store might only have access to their own information, etc.  I’m only showing the basics of this method in this article.  I’ll cover this subject more thoroughly in a future blog post.

Next in the code is the dummy lookup for the information requested.  If you navigate to my sample dummy data retriever you’ll see that I just check to see which product is requested and stuff a record in the list.  Obviously, this is the place where you’ll code a database select and insert records into the list from the database.

Next, the list of inventory records are serialized into a JSON format and then attached to the content of the response message.  This is then returned.

Next, you’ll need to setup an IIS server to serve your API.

 

Setting up the IIS Server

I’m going to setup an application in IIS 7 to show how to get this API working from your PC.  Eventually, you’ll want to setup an IIS server on your destination server that you will be deploying this application to.

I’ll be using IIS 7 for this demo, if you’re using Windows 7 or earlier, you will probably need to download and install IIS 7 (the express version works too).  When you open IIS 7 you should see a treeview on the left side of the console:


Right-click on “Default Web Site” and “Add Application“.  Name it “WebApiDemo“.  You should now see this:


Now click on the “WebApiDemo” node and you’ll see a panel on the right side of the control panel named “actions“.  Click on the “Basic Settings” link.  Change the physical path to point to your project location for the API project (this is the MVC4 project that you created earlier or downloaded from my Github account).



You’ll notice that the application pool is set to “DefaultAppPool“.  We’ll need to change this to a .Net 4 pool.  So click on the “Application Pools” node and double-click on the “DefaultAppPool” line.  Change the .Net Framework Version to version 4:


At this point, if you try to access your API, you’ll discover that it doesn’t work.  That’s because we need to give IIS server access to your project directory.  So navigate to your project directory, right-click and go to properties, then click on the “Security” tab.  Click the “advanced” button and “Change Permissions” button.  Then click the “Add” button.  The first user you’ll need to add will be the IIS_IUSRS permission.  Click the “Check Names” button and it should add your machine name as the domain of this user (my machine name is “FRANK-PC”, yours will be different):


You’ll need to give IIS permissions.  I was able to make the API work with these permissions:

I would recommend doing more research before setting up your final API server.  I’m not going to cover this subject in this post.

Now click “OK”, “OK”, “OK”, etc.

Now run through those steps again: to add a user named IUSR, with the same permissions:




If you didn’t setup your URL in the hosts file, you’ll need to do that now. If you did this in the retriever section above, then you can skip this step.  

Navigate to C:WindowsSystem32driversetc and edit the “hosts” file with a text editor.  You’ll see some sample text showing the format of a URL.  Create a domain name on a new line like this:

127.0.0.1        www.franksdomain.com

Remember, you can makeup your own domain name.

Now, let’s test the site.  First, we need to determine what the URL will be when we access our API.  The easy method is to go into the IIS control panel and click on the “Browse urlname.com on *80 (http)” link:

Now you can copy the URL in the browser that popped up:

http://www.franksdomain.com/WebApiDemo


This URL is actually the URL to the MVC website in your application.  In order to access your API, your going to have to add to this URL:

http://www.franksdomain.com/WebApiDemo/api/MyApi

How did I get the “api/MyApi“?  If you go to your MVC application and navigate to the “App_Start/WebApiConfig.cs” file, you’ll see this default setup:

config.Routes.MapHttpRoute(
        name: “DefaultApi“,
        routeTemplate: “api/{controller}/{id}“,
        defaults: new { id = RouteParameter.Optional }
);



So the URL contains “api” and then the controller name.  Remember the previous code for the API controller:

public class MyApiController : ApiController

Ignore the “Controller” part of the class name and you’ll have the rest of your path.  Put this path string in the app.config file of your retriever application and compile both applications.  

Startup fiddler.  Run your retriever.  Your retriever should run through without any errors (unless you or I missed a step).  In fiddler, select JSON for the top right and bottom right panes. You should see something like this:

Notice that the retriever sent a request for product ids 2,3,4 and the API returned detailed product information for those three products (Spoon, Fork and Knife).  You might need to expand the tree-views in your JSON panels to see all the information like that shown in the screen-shot above.


Setting a Breakpoint in the API Application


Your API application doesn’t really run until it is accessed by IIS.  So we need to run the retriever once and then we can set our debugger to attach to the application process.  Once you run your retriever, go to your API application, in Visual Studio click on the “Debug” menu, then select “attach to process“.  Then make sure the “Show processes from all users” is checked at the bottom of the dialog box.  Then find the process named “w3wp.exe” and click on it. 



Click the “attach” button.  Then click the “attach” button in the pop-up dialog.  You’ll notice that your application is now “running” (you can tell by the red stop button in your tool bar):


Put a break-point in your code.  Run the retriever program and you see your API project pop-up with your break-point waiting for you.  Now you can step through your program and inspect variables (such as the request variables that were received, hint, hint) just like you ran this program using the F-5 key.



Summary

There are a lot of little things that must occur correctly for this API to work properly.  Here are a few things to look out for:

– Your firewall could also block requests, so be on the lookout for that problem.  
– If you don’t have the correct URL setup in your retriever, you will not get a connection.  
– The objects you use to serialize and de-serialize JSON data must match between your retriever and API applications.  
– The IIS server must have the correct rights to the API application directory.  
– You need to setup the correct .NET type in your IIS application pool.
– Verify that IIS is running on your PC.
– Verify your inputs on your API by attaching to the w3wp.exe process and breaking on the first line of code.
– Verify your output from your retriever using fiddler.

Getting the Sample Projects

You can download the two sample projects from my GitHub account:  
https://github.com/fdecaire/WebApiDemo 

 

Leave a Reply