This exception comes due to simple reason that there are MIME Types missing for your silverlight Application which gives you following exception in your IE,

Silverlight Object Exception

Silverlight Object Exception

 

 

 

 

To reslove the same you simply need to add the MIME types to your WebSite on IIS. Select HTTP Headers Tab under the Properties. Click on MIME Types and add

   .xaml            application/xaml+xml

  .xap               application/octet-stream

You also need to enable the Content Expiration

Now your silverlight object will be loaded. If you still face problems try checking up on your permissions