Launch of Yemen’s first multi-source news crawler [Archives:2007/1053/Local News]

May 21 2007

SANA'A, May 23 ) Keeping up with the advances in Internet technology and the opportunities it opens for Yemeni Media, the first multi-source news crawling and search engine has been launched in order to create a one-stop news source for all Yemeni and related news from several internet news sites in Arabic and English. is a news gathering and sorting system similar to except that it is dedicated to Yemen. The portal is currently exclusively assigned to crawl and index news items from the dozens of Yemeni news websites, but will in the future expand to include other sections such as business, tourism, and community-related services.

Engineer Walid Al-Saqaf, co-founder and manager of the news crawler stated that “the goal out of this project is to inform the people of what is going on in Yemen and how information is represented by all different news sources which may be governmental, in opposition, independent, Arab, and International. The Internet's role as a medium to receive information will no doubt become more significant in the future for Yemen and the world. Therefore, we decided to bring together a search engine that crawls all Arabic and English major news outlets in Yemen and present them in a searchable and informative matter.”

The News crawler is also able to extract headlines from news website that are being blocked by the authorities, including updated and timely news from opposition news sites such as and Commentators has stated that this crawler will not only expand the span of press freedoms in Yemen through providing a single website which automatically publishes the latest news from government, independent, opposition, and international news websites, but it will also provide a way around government censorship inside Yemen which restricts several opposition news websites.

Media consultant and Co-founder of the website Ms. Irene Knehtl has emphasized the need to encourage online news sites that “have taken the burden of providing the public with news and quality content voluntarily, and their efforts need to be appreciated.” Knehtl noted that the Yemeni news websites currently in the system includes independent, opposition, and governmental sources. “We are still in our initial phase and are expanding, we are expected to add more sources as we go along and welcome feedback from users' and sources.”

According to Al-Saqaf, applies the latest publicly advanced technology available of retrieving, parsing, and indexing information on a frequent basis depending on the time of the day and the flow of information. It has a flexible dynamic interface that allows users to maximize their benefit by displaying news items in a customizable fashion, while remaining faithful to the copyright material as it maintains links to the sources and promotes them through exposure to new or unfamiliar users.

He also added that the portal is “merely the first in a series of phases that we are going to take. The next phase will be to create a forum section in the portal where comments and contributions in popular forums will be crawled and indexed. Following that will be a phase to include tourism-related information and contacts to serve cyber users with as a bridge for all those interested in Yemen.”