Offline Category DatabaseLocally hosted domain categorization database
Our offline database is perfect for customers who require a local option for domain categorization. Typical use cases include hardware/product integrations, custom applications or internal use at an Enterprise organization with high request volume.
In addition to flexible pricing options for production uses, we offer affordable pricing for startups or organizations looking to prototype or bring a product.
The Webshrinker database is typically provided in JSON format, and updates are able to be downloaded directly from Amazon S3 as JSON files as often as desired. Alternative delivery methods include an HTTPs URL callback or immediate delivery to an Amazon SQS of your choice. These incremental updates may be applied via the database back end of your choosing, with most customers importing into MongoDB, Postgres or MySQL.
Data Feed HighlightsA powerhouse - perfect for use in high volume or custom applications!
Big Data
Using advanced machine learning, we categorize billions of web pages, keeping the categorization database one of the most accurate in the industry.
Wide Range of Applications
Our domain categorization allows you to easily provide services such as Internet filtering, subscriber analytics, advertising networks, and fraud prevention.
Data Firehose
Need access to the raw data stream of everything as it’s being categorized? We provide a real-time data firehose via HTTP, Amazon S3, or Amazon SQS.
Advanced Algorithms
Our technology categorizes content found on URLs, as well as entire websites and IP addresses – perfect for security devices that may not have access to the full URL.
Developer-Friendly Responses
Every API response is returned as JSON, which can be easily read and implemented into your product.
Always Up to Date
Our web crawlers are constantly visiting and classifying new and existing websites, providing real-time results and keeping the database always up to date.
Easily Block Bad Content
You can easily block bad content within your application, appliance or company network. Simply query our API with the requested URL and reject if the category is not part of your “approved” list.
Infinitely Scalable
Running on Amazon Web Services, our APIs are easily scalable to support any growth or requirements.
Top Tier Support
Your happiness is our success. Our support team is compromised of real engineers, here to help with any challenge or need.
Sample Database JSON Output
{
"_id": "ceb243bac6752229b65acc3d0443d7bb7306343d5818d4c1bb9f681f9dec2df7",
"categories": [
"informationtech",
"business"
]
}Available Category StructuresWe offer two top level category structures to chose from
Whether you need the deep categorization provided by the IAB taxonomy or prefer a more simple category structure, we’ve got you covered. You can choose between our implementation of IAB categories (390+ categories) or the Web Shrinker standard categories (42 categories) at any time.
ABORTION
Sites which provide views either in favor or against abortion, provide details on procedures, offer help or discuss outcomes or consequences of abortion.
ADULT CONTENT
Sites which may contain sexually explicit content, images, or that are portrayed through visually expressive language.
ADVERTISING
Sites or businesses which directly sell ads to consumers through various mediums, including Internet, TV, or radio.
ALCOHOL AND TOBACCO
Sites that sell, discuss or glorify the consumption of various alcoholic and tobacco products, including beer, wine, and liquor.
BLOGS AND PERSONAL SITES
Includes sites that make use of common blogging software including WordPress, Joomla!, and Drupal, amongst others, which generate dynamic content. Normally used as a secondary category to a more descriptive primary category.
BUSINESS
Sites which exhibit business-like attributes such as selling of services, products, or consulting. Normally used as a secondary category to a more descriptive primary category.
CHAT AND INSTANT MESSAGING
Sites which provide chat or text messaging services or such abilities through a download or application.
CONTENT SERVERS
Sites whose main purpose is to serve static images, CSS and JavaScript files.
DATING AND PERSONALS
Sites whose main focus is on connecting individuals for the purposes of dating.
DECEPTIVE
Sites that attempt to trick the user into believing they are on a different site in order to gather information or for other purposes. Also includes sites with deceptive advertising practices such as performing click redirections without the user’s consent.
DRUGS
Sites that contain content whose main focus is on controlled substances, including the sale, discussion, or glorification of such substances. Does not include alcohol and tobacco as that has its own category.
ECONOMY AND FINANCE
Includes sites that are mainly focused on stocks and current market information or provide financial services such as banks or lenders.
EDUCATION AND SELF-HELP
Sites whose main purpose is to offer educational information, community information, or how-tos. Also includes educational facilities and related organizations.
ENTERTAINMENT
Sites that focus on art and entertainment, including topics like TV/Hollywood, tattoos, cartoons, and anime.
FOOD AND RECIPES
Sites that contain food-related information, recipes, food preparation or restaurant services.
GAMBLING
Sites that allow a visitor to play games using wagers/placing bets, lottery pools, or provides information on such activities.
GAMES
Sites that provide games, including online games or through an application.
HACKING AND CRACKING
Sites that disseminate information, hold discussions or provide a means to gain unauthorized or illegal access to computers and networks.
HEALTH
Sites whose content is focused on a person’s well-being, including fitness or workout information, medical conditions, diagnoses, and medical services.
HUMOR
Sites that contain content with a focus on jokes or comedy, including satire.
ILLEGAL CONTENT
Sites that focus on providing links to pirated movies, commercial software, or providing application keys and cracks for commercial applications.
INFORMATION TECHNOLOGY
Sites whose main focus is on computers or distributing computer related information, including computer networking, Internet telephony, operating systems, or programming.
JOBS AND CAREERS
Sites that focus on assisting visitors in finding employment, career guidance or improvement.
MALICIOUS
Sites that are infected with or distribute any kind of malware, spyware, or viruses. Also contains sites acting as a C&C for bots, worms, trojans, and other malware.
MEDIA SHARING
Sites that allow visitors to upload content and share media such as photos and videos.
MESSAGE BOARDS AND FORUMS
Sites which provide some type of a messaging or bulletin board system whose content is largely community generated.
NEWS AND MEDIA
Sites whose content is mostly focused on current events and topics. Includes various news outlets, radio, TV stations, and magazines.
PARKED SITES AND DOMAINS
Sites/domain names that are no longer controlled by the original owner or are being offered for sale. Content on these sites can often be misleading for non-technical users.
PROXY AND FILTER AVOIDANCE
Sites that provide information or a means to circumvent filtering proxies or detection systems, including VPN services and anonymous surfing.
REAL ESTATE
Sites that provide information or services for renting, selling, or buying property.
RELIGION
Sites that provide information on one or more religious beliefs, practices, or are affiliated with a religious institution such as a church or synagogue.
SEARCH ENGINES AND PORTALS
Sites that enable their visitors to search the Internet or whose main focus is to provide links to other Internet sites.
SHOPPING
Sites that sell products or services, normally with an online purchasing interface.
SOCIAL NETWORKING
Sites that provide a community portal whereby members join and contribute posts or media and forge connections with other members.
STREAMING MEDIA
Sites that are mainly dedicated to the serving of video or audio streams and downloads.
SPORTS
Sites that contain information about various sports and sporting activities, including sports scores or team information.
TRANSLATION SITES
Sites that perform translation from one language to another, usually performed by a computer. May also include personal translation services.
TRAVEL
Sites that focus on travel planning services, travel reservations, and tourist information.
UNCATEGORIZED
Sites that are not currently classified as belonging to one of the other categories or are not yet classified.
VEHICLES
Sites that mainly hold discussions or share information about vehicles, including cars, trucks, boats, and aircraft.
VIRTUAL REALITY
Sites that host files specific to virtual reality or run communities related to the technology.
WEAPONS
Sites that primarily discuss, review, or sell items such as hunting knives, guns, rifles, or BB guns.
IAB1 – ARTS & ENTERTAINMENT
- IAB1-1 Books & Literature
- IAB1-2 Celebrity Fan/Gossip
- IAB1-3 Fine Art
- IAB1-4 Humor
- IAB1-5 Movies
- IAB1-6 Music & Audio
- IAB1-7 Television & Video
IAB2 – AUTOMOTIVE
- IAB2-1 Auto Parts
- IAB2-2 Auto Repair
- IAB2-3 Buying/Selling Cars
- IAB2-4 Car Culture
- IAB2-5 Certified Pre-Owned
- IAB2-6 Convertible
- IAB2-7 Coupe
- IAB2-8 Crossover
- IAB2-9 Diesel
- IAB2-10 Electric Vehicle
- IAB2-11 Hatchback
- IAB2-12 Hybrid
- IAB2-13 Luxury
- IAB2-14 Minivan
- IAB2-15 Motorcycles
- IAB2-16 Off-Road Vehicles
- IAB2-17 Performance Vehicles
- IAB2-18 Pickup
- IAB2-19 Road-Side Assistance
- IAB2-20 Sedan
- IAB2-21 Trucks & Accessories
- IAB2-22 Vintage Cars
- IAB2-23 Wagon
IAB3 – BUSINESS
- IAB3-1 Advertising
- IAB3-2 Agriculture
- IAB3-3 Biotech/Biomedical
- IAB3-4 Business Software
- IAB3-5 Construction
- IAB3-6 Forestry
- IAB3-7 Government
- IAB3-8 Green Solutions
- IAB3-9 Human Resources
- IAB3-10 Logistics
- IAB3-11 Marketing
- IAB3-12 Metals
IAB4 – CAREERS
- IAB4-1 Career Planning
- IAB4-2 College
- IAB4-3 Financial Aid
- IAB4-4 Job Fairs
- IAB4-5 Job Search
- IAB4-6 Resume Writing/Advice
- IAB4-7 Nursing
- IAB4-8 Scholarships
- IAB4-9 Telecommuting
- IAB4-10 Military
- IAB4-11 Career Advice
IAB5 – EDUCATION
- IAB5-1 7-12 Education
- IAB5-2 Adult Education
- IAB5-3 Art History
- IAB5-4 College Administration
- IAB5-5 College Life
- IAB5-6 Distance Learning
- IAB5-7 English as a 2nd Language
- IAB5-8 Language Learning
- IAB5-9 College / Graduate School
- IAB5-10 Homeschooling
- IAB5-11 Homework/Study Tips
- IAB5-12 K-6 Education
- IAB5-13 Private School
- IAB5-14 Special Education
- IAB5-15 Studying Business
IAB6 – FAMILY & PARENTING
- IAB6-1 Adoption
- IAB6-2 Babies & Toddlers
- IAB6-3 Daycare/Pre School
- IAB6-4 Family Internet
- IAB6-5 Parenting – K-6 Kids
- IAB6-6 Parenting teens
- IAB6-7 Pregnancy
- IAB6-8 Special Needs Kids
- IAB6-9 Eldercare
IAB7 – HEALTH & FITNESS
- IAB7-1 Exercise / Weight Loss
- IAB7-2 ADD
- IAB7-3 AIDS / HIV
- IAB7-4 Allergies
- IAB7-5 Alternative Medicine / Holistic Healing
- IAB7-6 Arthritis
- IAB7-7 Asthma
- IAB7-8 Autism / PDD / Asperger’s Syndrome
- IAB7-9 Bipolar Disorder
- IAB7-10 Brain Tumor
- IAB7-11 Cancer
- IAB7-12 Cholesterol
- IAB7-13 Chronic Fatigue Syndrome
- IAB7-14 Chronic Pain
- IAB7-15 Cold & Flu
- IAB7-16 Deafness
- IAB7-17 Dental Care
- IAB7-18 Depression
- IAB7-19 Dermatology
- IAB7-20 Diabetes
- IAB7-21 Epilepsy / Seizures
- IAB7-22 GERD / Acid Reflux
- IAB7-23 Headaches / Migraines / Fevers / Pain
- IAB7-24 Heart Disease / Stroke
- IAB7-25 Herbs for Health / Supplements / Vitamins
- IAB7-26 Holistic Healing
- IAB7-27 IBS / Crohn’s / Celiac Disease
- IAB7-28 Incest / Abuse Support
- IAB7-29 Incontinence
- IAB7-30 Infertility
- IAB7-31 Men’s Health
- IAB7-32 Nutrition
- IAB7-33 Orthopedics
- IAB7-34 Panic / Anxiety Disorders
- IAB7-35 Pediatrics
- IAB7-36 Physical Therapy
- IAB7-37 Psychology / Psychiatry / Therapy
- IAB7-38 Senior Health
- IAB7-39 Sexuality
- IAB7-40 Sleep Disorders
- IAB7-41 Smoking Cessation
- IAB7-42 Substance Abuse
- IAB7-43 Thyroid Disease / Endocrinology
- IAB7-44 Weight Loss
- IAB7-45 Women’s Health
- IAB7-WS1 Abortion
IAB8 – FOOD & DRINK
- IAB8-1 American Cuisine
- IAB8-2 Barbecues & Grilling
- IAB8-3 Cajun / Creole
- IAB8-4 Chinese Cuisine
- IAB8-5 Cocktails / Beer
- IAB8-6 Coffee / Tea
- IAB8-7 Cuisine-Specific
- IAB8-8 Desserts & Baking
- IAB8-9 Dining Out
- IAB8-10 Food Allergies
- IAB8-11 French Cuisine
- IAB8-12 Health / Low-Fat Cooking
- IAB8-13 Italian Cuisine
- IAB8-14 Japanese Cuisine
- IAB8-15 Mexican Cuisine
- IAB8-16 Vegan
- IAB8-17 Vegetarian
- IAB8-18 Wine
IAB9 – HOBBIES & INTERESTS
- IAB9-1 Art / Technology
- IAB9-2 Arts & Crafts
- IAB9-3 Beadwork
- IAB9-4 Bird-Watching
- IAB9-5 Board Games / Puzzles
- IAB9-6 Candle & Soap Making
- IAB9-7 Card Games
- IAB9-8 Chess
- IAB9-9 Cigars / Vaping / Tobacco & Accessories
- IAB9-10 Collecting
- IAB9-11 Comic Books / Anime / Manga
- IAB9-12 Drawing / Sketching
- IAB9-13 Freelance Writing / Getting Published
- IAB9-14 Genealogy
- IAB9-15 Getting Published
- IAB9-16 Guitar / Keyboard / Drums
- IAB9-17 Home Recording
- IAB9-18 Investors / Inventors / Patents / Copyright
- IAB9-19 Jewelry Making
- IAB9-20 Magic & Illusion
- IAB9-21 Needlework
- IAB9-22 Painting
- IAB9-23 Photography
- IAB9-24 Radio
- IAB9-25 Roleplaying Games
- IAB9-26 Sci-Fi & Fantasy
- IAB9-27 Scrapbooking
- IAB9-28 Screenwriting
- IAB9-29 Stamps & Coins
- IAB9-30 Video & Computer Games
- IAB9-31 Woodworking
- IAB9-WS1 Gambling
- IAB9-WS2 Weapons
IAB10 – HOME & GARDEN
- IAB10-1 Appliances
- IAB10-2 Entertaining
- IAB10-3 Environmental Safety
- IAB10-4 Gardening
- IAB10-5 Home Repair
- IAB10-6 Home Theater
- IAB10-7 Interior Decorating
- IAB10-8 Landscaping
- IAB10-9 Remodeling & Construction
IAB11 – LAW, GOVERNMENT, & POLITICS
- IAB11-1 Immigration
- IAB11-2 Legal Issues
- IAB11-3 Government Resources
- IAB11-4 Politics
- IAB11-5 Commentary
IAB12 – NEWS / WEATHER / INFORMATION
- IAB12-1 International News
- IAB12-2 National News
- IAB12-3 Local News
IAB13 – PERSONAL FINANCE
- IAB13-1 Beginning Investing
- IAB13-2 Credit / Debit & Loans
- IAB13-3 Financial News
- IAB13-4 Financial Planning
- IAB13-5 Hedge Fund
- IAB13-6 Insurance
- IAB13-7 Investing
- IAB13-8 Mutual Funds / ETFs
- IAB13-9 Options
- IAB13-10 Retirement Planning
- IAB13-11 Stocks
- IAB13-12 Tax Planning / Accounting
IAB14 – SOCIETY
- IAB14-1 Dating / Personals
- IAB14-2 Divorce Support
- IAB14-3 LGBTQ+
- IAB14-4 Marriage
- IAB14-5 Senior Living
- IAB14-6 Teens
- IAB14-7 Weddings
- IAB14-8 Ethnic Specific
- IAB14-WS1 Social Networking
IAB15 – SCIENCE
- IAB15-1 Astrology
- IAB15-2 Biology
- IAB15-3 Chemistry
- IAB15-4 Geology
- IAB15-5 Paranormal Phenomena
- IAB15-6 Physics
- IAB15-7 Space / Astronomy
- IAB15-8 Geography
- IAB15-9 Botany
- IAB15-10 Weather
IAB16 – PETS
- IAB16-1 Aquariums
- IAB16-2 Birds
- IAB16-3 Cats
- IAB16-4 Dogs
- IAB16-5 Large Animals
- IAB16-6 Reptiles
- IAB16-7 Veterinary Medicine
IAB17 – SPORTS
- IAB17-1 Auto Racing
- IAB17-2 Baseball / Softball
- IAB17-3 Bicycling
- IAB17-4 Bodybuilding
- IAB17-5 Boxing
- IAB17-6 Canoeing / Kayaking
- IAB17-7 Cheerleading
- IAB17-8 Climbing
- IAB17-9 Cricket
- IAB17-10 Figure Skating
- IAB17-11 Fly Fishing
- IAB17-12 American Football
- IAB17-13 Freshwater Fishing
- IAB17-14 Game & Fish
- IAB17-15 Golf
- IAB17-16 Horse Racing
- IAB17-17 Horses
- IAB17-18 Hunting / Shooting
- IAB17-19 Inline Skating
- IAB17-20 Martial Arts
- IAB17-21 Mountain Biking
- IAB17-22 NASCAR Racing
- IAB17-23 Olympics
- IAB17-24 Paintball
- IAB17-25 Power & Motorcycles
- IAB17-26 Basketball
- IAB17-27 Ice Hockey
- IAB17-28 Rodeo
- IAB17-29 Rugby
- IAB17-30 Running / Jogging
- IAB17-31 Sailing / Boating
- IAB17-32 Saltwater Fishing
- IAB17-33 Scuba Diving
- IAB17-34 Skateboarding
- IAB17-35 Skiing
- IAB17-36 Snowboarding
- IAB17-37 Surfing / Body-Boarding
- IAB17-38 Swimming
- IAB17-39 Table Tennis / Ping-Pong
- IAB17-40 Tennis
- IAB17-41 Volleyball
- IAB17-42 Walking
- IAB17-43 Waterski / Wakeboard
- IAB17-44 World Football / Soccer
IAB18 – STYLE & FASHION
- IAB18-1 Beauty
- IAB18-2 Body Art
- IAB18-3 Fashion
- IAB18-4 Jewelry
- IAB18-5 Clothing
- IAB18-6 Accessories
IAB19 – TECHNOLOGY & COMPUTING
- IAB19-1 3-D Graphics
- IAB19-2 Animation
- IAB19-3 Antivirus Software
- IAB19-4 C / C++
- IAB19-5 Cameras & Camcorders
- IAB19-6 Cell Phones
- IAB19-7 Computer Certification
- IAB19-8 Computer Networking
- IAB19-9 Computer Peripherals
- IAB19-10 Computer Reviews
- IAB19-11 Data Centers
- IAB19-12 Databases
- IAB19-13 Desktop Publishing
- IAB19-14 Desktop Video
- IAB19-15 Email / Chat / Messaging
- IAB19-16 Graphics Software
- IAB19-17 Home Video / DVD
- IAB19-18 Internet Technology
- IAB19-19 Java
- IAB19-20 JavaScript
- IAB19-21 Mac Support
- IAB19-22 MP3 / MIDI
- IAB19-23 Net Conferencing
- IAB19-24 Net for Beginners
- IAB19-25 Network Security
- IAB19-26 Palmtops / PDAs / Tablets
- IAB19-27 PC Support
- IAB19-28 Portable
- IAB19-29 Entertainment / Virtual & Augmented Reality
- IAB19-30 Shareware / Freeware / Open Source
- IAB19-31 Unix / Linux
- IAB19-32 Visual Basic
- IAB19-33 Web Clip Art
- IAB19-34 Web Design / HTML
- IAB19-35 Web Search
- IAB19-36 Windows
- IAB19-WS1 Hacking / Cracking
- IAB19-WS2 VPNs / Proxies & Filter Avoidance
- IAB19-WS3 Language Translation
- IAB19-WS4 File Sharing
IAB20 – TRAVEL
- IAB20-1 Adventure Travel
- IAB20-2 Africa
- IAB20-3 Air Travel
- IAB20-4 Australia & New Zealand
- IAB20-5 Bed & Breakfast
- IAB20-6 Budget Travel
- IAB20-7 Business Travel
- IAB20-8 By US Locale
- IAB20-9 Camping
- IAB20-10 Canada
- IAB20-11 Caribbean
- IAB20-12 Cruises
- IAB20-13 Eastern Europe
- IAB20-14 Europe
- IAB20-15 France
- IAB20-16 Greece
- IAB20-17 Honeymoons / Getaways
- IAB20-18 Hotels
- IAB20-19 Italy
- IAB20-20 Japan / China
- IAB20-21 Mexico & Central America
- IAB20-22 National Parks
- IAB20-23 South America
- IAB20-24 Spas
- IAB20-25 Theme Parks
- IAB20-26 Traveling with Kids
- IAB20-27 United Kingdom
IAB21 – REAL ESTATE
- IAB21-1 Apartments
- IAB21-2 Architects
- IAB21-3 Buying / Selling Homes
IAB22 – SHOPPING
- IAB22-1 Contests & Freebies
- IAB22-2 Couponing
IAB23 – RELIGION & SPIRITUALITY
- IAB23-1 Alternative Religions
- IAB23-2 Atheism / Agnosticism
- IAB23-3 Buddhism
- IAB23-4 Catholicism
- IAB23-5 Christianity
- IAB23-6 Hinduism
- IAB23-7 Islam
- IAB23-8 Judaism
- IAB23-9 Latter-Day Saints
- IAB23-10 Pagan / Wiccan
IAB24 – UNCATEGORIZED
IAB25 – NON-STANDARD CONTENT
- IAB25-1 Unmoderated UGC / Message Boards
- IAB25-2 Extreme Graphic / Explicit Violence
- IAB25-3 Adult Content
- IAB25-4 Profane Content
- IAB25-5 Hate Content
- IAB25-6 Under Construction
- IAB25-7 Incentivized
- IAB25-WS1 Content Server
- IAB25-WS2 Streaming Media
IAB26 – ILLEGAL CONTENT
- IAB26-1 Illegal Content
- IAB26-2 Warez
- IAB26-3 Spyware / Malware / Malicious
- IAB26-4 Copyright Infringement
- IAB26-WS1 Illegal Drugs & Paraphernalia
- IAB26-WS2 Deceptive / Phishing