HTML article content extraction
I've been doing a lot of research to figure out the best way to code an application to get the main article content from almost any HTML webpage. I have a C program that uses libxml2 to parse through the XML, but I came across Alchemy API, which appears to do what I want.
However, it only has an online API and I wanted to keep the application in-house without relying on any external calls.
So does anybody have tips? I was hoping for an off-line alternative that does what Alchemy API can do (paid/non-paid).
My alternative may be to just parse the HTML and use NLP (Natural Language Processing) techniques and other methods to get at the main article content. The types of websites that it will be used include websites with a news section or a blog.
there are a few open source tools available that do similar article extraction tasks. https://github.com/jiminoc/goose which was open source by Gravity.com
It has info on the wiki as well as the source you can view. There are dozens of unit tests that show the text extracted from various articles.
AlchemyAPI also offers an on-premise solution so that you don't have to access it online. Generally our customers that have the on-premise solutions are using it if they have special security or latency requirements. More information on on-premise solutions can be found here: http://www.alchemyapi.com/products/on-premise/
链接地址: http://www.djcxy.com/p/48752.html上一篇: ffmpeg和opengl
下一篇: HTML文章内容提取