Offline URL Category Database - Web Shrinker

Offline URL Category Database

Offline URL Category Database

Offline Website URL Category Database

Integrate our category database into your behind-the-firewall solution

Build your solution on top of our database

Does your product or company policy require offline categorization? No problem, we offer an offline option with our category database, allowing you to work without calling the API for each request.

Contact us for more information about the database

Vast amount of data

With a single query, get the same category results that we provide through our Website Category API service right from your local database store.

Easily block bad content

You can easily block bad websites within your company network without making API calls to our system for each request. Simply run a lookup of the requested URL and reject it if the web page belongs to a "bad" category.

Built for speed

Incremental database updates are provided continuously, in real time via Amazon S3 JSON files. You can download the updates as often as you like, the intervals are user-defined so that you have full control over it.

Developer friendly structure

The database is packed in JSON files that can be easily imported into popular database systems like MySQL and MongoDB. All you have to do is to run a simple import command and the system will do the rest for you.

Many types of distinct categories

Currently, we have 40 categories in our system and are constantly adding new ones to the list. If you are looking for a certain category that we don’t currently have, feel free to send us a feature request.

Always up-to-date

Our web crawlers are constantly visiting and classifying both new and existing web sites, providing you with real-time results and keeping the database updated with the latest information.

First class support

Your happiness is our success. We’re here to help you with any kind of question and help you with your growing needs.

Sample Database JSON

 { 
"_id": "ceb243bac6752229b65acc3d0443d7bb7306343d5818d4c1bb9f681f9dec2df7",
"categories": [
"informationtech",
"business"
]
}

Contact us for more information about the database

Current Website Categories

ABORTION

Sites which provide views either in favor or against abortion, provide details on procedures, offer help or discuss outcomes or consequences of abortion.

ADULT CONTENT

Sites which may contain sexually explicit content, images, or that are portrayed through visually expressive language.

ADVERTISING

Sites or businesses which directly sell ads to consumer through various mediums, including Internet, TV, or radio.

ALCOHOL AND TOBACCO

Sites that sell, discuss, or glorify the consumption of various alcoholic and tobacco products, including beer, wine, and liquor.

BLOGS AND PERSONAL SITES

Includes sites that make use of common blogging software including WordPress, Joomla!, and Drupal, amongst others, which generate dynamic content. Normally used as a secondary category to a more descriptive primary category.

BUSINESS

Sites which exhibit business-like attributes such as selling of services, products, or consulting. Normally used as a secondary category to a more descriptive primary category.

CHAT AND INSTANT MESSAGING

Sites which provide chat or text messaging services or such abilities through a download or application.

CONTENT SERVERS

Sites whose main purpose is to serve static image, CSS, and JavaScript files.

DATING AND PERSONALS

Sites whose main focus is on connecting individuals for the purposes of dating.

DRUGS

Sites that contain content whose main focus is on controlled substances, including the sale, discussion, or glorification of such substances. Does not include alcohol and tobacco as that has its own category.

ECONOMY AND FINANCE

Includes sites that are mainly focused on stocks and current market information or provide financial services such as banks or lenders.

EDUCATION AND SELF-HELP

Sites whose main purpose is to offer educational information, community information, or how-tos. Also includes educational facilities and related organizations.

ENTERTAINMENT

Sites that focus on art and entertainment, including topics like TV/Hollywood, tattoos, cartoons and anime.

FOOD AND RECIPES

Sites that contain food related information or recipes, food preparation, or restaurant services.

GAMBLING

Sites that allow a visitor to play games using wagers/placing bets, lottery pools, or provides information on such activities.

GAMES

Sites that provide games, including online games or through an application.

HACKING AND CRACKING

Sites that disseminate information, hold discussions, or provide a means to gain unauthorized or illegal access to computers and networks.

HEALTH

Sites whose content is focused on a person's well-being, including fitness or workout information, medical conditions, diagnoses, and medical services.

HUMOR

Sites that contain content with a focus on jokes or comedy, including satire.

ILLEGAL CONTENT

Sites that focus on providing links to pirated movies, commercial software, or providing application keys and cracks for commercial applications.

INFORMATION TECHNOLOGY

Sites whose main focus is on computers or distributing computer related information, including computer networking, Internet telephony, operating systems, or programming.

JOBS AND CAREERS

Sites that focus on assisting visitors in finding employment, career guidance or improvement.

MEDIA SHARING

Sites that allow visitors to upload content and share media such as photos and videos.

MESSAGE BOARDS AND
FORUMS

Sites which provide some type of a messaging or bulletin board system whose content is largely community generated.

NEWS AND MEDIA

Sites whose content is mostly focused on current events and topics. Includes various news outlets, radio, TV stations, and magazines.

PARKED SITES AND DOMAINS

Sites/domain names that are no longer controlled by the original owner or are being offered for sale. Content on these sites can often be misleading for non technical users.

PROXY AND FILTER AVOIDANCE

Sites that provides information or a means to circumvent filtering proxies or detection systems, including VPN services and anonymous surfing.

REAL ESTATE

Sites that provide information or services for renting, selling, or buying property.

RELIGION

Sites that provide information on one or more religious beliefs, practices, or are affiliated with a religious institution such as a church or synagogue.

SEARCH ENGINES AND
PORTALS

Sites that enable their visitors to search the Internet or whose main focus is to provide links to other Internet sites.

SHOPPING

Sites that sell products or services, normally with an online purchasing interface.

SOCIAL NETWORKING

Sites that provide a community portal whereby members join and contribute posts or media and forge connections with other members.

SPORTS

Sites that contain information about various sports and sporting activities, including sports scores or team information.

STREAMING MEDIA

Sites that are mainly dedicated to the serving of video or audio streams and downloads.

TRANSLATION SITES

Sites that perform translation from one language to another, usually performed by a computer. May also include personal translation services.

TRAVEL

Sites that focus on travel planning services, travel reservations, and tourist information.

UNCATEGORIZED

Sites that are not currently classified as belonging to one of the other categories or are not yet classified.

VEHICLES

Sites that mainly hold discussions or share information about vehicles, including cars, trucks, boats, and aircraft.

VIRTUAL REALITY

Sites that host files specific to virtual reality or run communities related to the technology.

WEAPONS

Sites that primarily discuss, review, or sell items such as hunting knives, guns, rifles, or BB guns.

Contact us for more information about the database