Skip to content Skip to sidebar Skip to footer

Html-agility-pack Not Loading The Page With Full Content?

i am using Html Agility Pack to fetch data from website(scrapping) My problem is the website from i am fetching the data is load some of the content after few seconds of page load.

Solution 1:

It's not about delaying the request. That node is populated by javascript using the DOM and the Html Agility Pack is the wrong tool for that requirement (it isn't a web engine at all, it only loads the base Html).

When I need to get at stuff that requires a full web engine to parse, I typically use WatiN. It's designed to help unit test actual web pages, but that means it allows programmatic access to web pages through a given browser engine and will load the full document. It comes with IE or Firefox drivers out of the box and I vaguely recall that Chrome wasn't hard to use, either.

Post a Comment for "Html-agility-pack Not Loading The Page With Full Content?"