Htmlagilitypack Query Returning No Value

Been struggling for 2 days. I'm using C# and HtmlAgilityPack within a .NET 4.5 winforms project to extract data from a website (the field I want to extract is $ flow and B/S ratio)

Solution 1:

Page you are querying does not contain any data in table with id th_data. If you will examine page markup, you'll see:

<table cellpadding="0" cellspacing="0" border="0"class="display" id="tf_data">
    <thead><trheight="10"><thalign="center"></th><thalign="center"width="90">CHART</th><thalign="left"width="70">SYMBOL</th><thalign="left">MARKET CAP</th><thalign="right"width="65">PRICE</th><thalign="center"width="80">CHANGE</th><thalign="right">VOL</th><thalign="right">B/S RATIO</th><thalign="right"width="80">NET CASH FLOW</th></tr></thead><tbody> <-- empty!
    </tbody></table>

All data are added to this table by browser via Java Script after document is loaded (see $(document).ready function). So if you are getting html from that url, there will be no data until browser will run Java Script code. I.e. there is nothing you can parse.

I suggest you to examine script which loads JSON data into page, and simply call same service from your code.

---

Its out of question scope, but for retrieving data you can use HttpClient class from System.Net.Http assembly. Here is sample of usage (its up to you to analyze how query string should be composed):

HttpClientclient=newHttpClient();
client.BaseAddress = newUri("http://finance.avafin.com");
stringurl="data?sEcho=2&iColumns=9&sColumns=&iDisplayStart=0&iDisplayLength=20&mDataProp_0=0&mDataProp_1=1&mDataProp_2=2&mDataProp_3=3&mDataProp_4=4&mDataProp_5=5&mDataProp_6=6&mDataProp_7=7&mDataProp_8=8&sSearch=&bRegex=false&sSearch_0=&bRegex_0=false&bSearchable_0=true&sSearch_1=&bRegex_1=false&bSearchable_1=true&sSearch_2=&bRegex_2=false&bSearchable_2=true&sSearch_3=&bRegex_3=false&bSearchable_3=true&sSearch_4=&bRegex_4=false&bSearchable_4=true&sSearch_5=&bRegex_5=false&bSearchable_5=true&sSearch_6=&bRegex_6=false&bSearchable_6=true&sSearch_7=&bRegex_7=false&bSearchable_7=true&sSearch_8=&bRegex_8=false&bSearchable_8=true&iSortCol_0=4&sSortDir_0=asc&iSortingCols=1&bSortable_0=true&bSortable_1=true&bSortable_2=true&bSortable_3=true&bSortable_4=true&bSortable_5=true&bSortable_6=true&bSortable_7=true&bSortable_8=true&type=BS_RATIO&date=06%2F14%2F2013&categoryName=&alertId=0&alertId2=&industryId=0&sectorId=0&symbol=spy&recom=&period=&perfPercent=";
varresponse= client.GetStringAsync(url).Result;

Response will contain html which you can parse.