3
wowotek
6y

For any webDev in here, is there any open source tools for web data extraction? i've seen imacros and UiPath, maybe you use other tools.

Comments
  • 1
    Scraping you mean?
  • 1
    @Codex404 is that the term? maybe yes.

    do you know any?
  • 1
    @wowotek if you mean downloading and parsing Html data then there's a nice Python library called Beautiful Soup
  • 1
    @wowotek Node is what I use for it, javascript is ideal for html reading.
  • 1
    if it’s dynamic content you need some headless browser ex. phantomjs and use node or install selenium driver for your favorite language.
    If site is static, (not common today) you can find library in almost any language.
Add Comment