Offline Category DatabaseLocally hosted domain categorization database
Our offline database is perfect for customers who require a local option for domain categorization. Typical use cases include hardware/product integrations, custom applications or internal use at an Enterprise organization with high request volume.
In addition to flexible pricing options for production uses, we offer affordable pricing for startups or organizations looking to prototype or bring a product.
The Webshrinker database is typically provided in JSON format, and updates are able to be downloaded directly from Amazon S3 as JSON files as often as desired. Alternative delivery methods include an HTTPs URL callback or immediate delivery to an Amazon SQS of your choice. These incremental updates may be applied via the database back end of your choosing, with most customers importing into MongoDB, Postgres or MySQL.
Data Feed HighlightsA powerhouse - perfect for use in high volume or custom applications!
Big Data
Using advanced machine learning, we categorize billions of web pages, keeping the categorization database one of the most accurate in the industry.
Wide Range of Applications
Our domain categorization allows you to easily provide services such as Internet filtering, subscriber analytics, advertising networks, and fraud prevention.
Data Firehose
Need access to the raw data stream of everything as it’s being categorized? We provide a real-time data firehose via HTTP, Amazon S3, or Amazon SQS.
Advanced Algorithms
Our technology categorizes content found on URLs, as well as entire websites and IP addresses – perfect for security devices that may not have access to the full URL.
Developer-Friendly Responses
Every API response is returned as JSON, which can be easily read and implemented into your product.
Always Up to Date
Our web crawlers are constantly visiting and classifying new and existing websites, providing real-time results and keeping the database always up to date.
Easily Block Bad Content
You can easily block bad content within your application, appliance or company network. Simply query our API with the requested URL and reject if the category is not part of your “approved” list.
Infinitely Scalable
Running on Amazon Web Services, our APIs are easily scalable to support any growth or requirements.
Top Tier Support
Your happiness is our success. Our support team is compromised of real engineers, here to help with any challenge or need.
Sample Database JSON Output
{ "_id": "ceb243bac6752229b65acc3d0443d7bb7306343d5818d4c1bb9f681f9dec2df7", "categories": [ "informationtech", "business" ] }
Available Category StructuresWe offer two top level category structures to chose from
Whether you need the deep categorization provided by the IAB taxonomy or prefer a more simple category structure, we’ve got you covered. You can choose between our implementation of IAB categories (390+ categories) or the Web Shrinker standard categories (42 categories) at any time.
ABORTION
ADULT CONTENT
ADVERTISING
ALCOHOL AND TOBACCO
BLOGS AND PERSONAL SITES
BUSINESS
CHAT AND INSTANT MESSAGING
CONTENT SERVERS
DATING AND PERSONALS
DECEPTIVE
DRUGS
ECONOMY AND FINANCE
EDUCATION AND SELF-HELP
ENTERTAINMENT
FOOD AND RECIPES
GAMBLING
GAMES
HACKING AND CRACKING
HEALTH
HUMOR
ILLEGAL CONTENT
INFORMATION TECHNOLOGY
JOBS AND CAREERS
Sites that focus on assisting visitors in finding employment, career guidance or improvement.
MALICIOUS
Sites that are infected with or distribute any kind of malware, spyware, or viruses. Also contains sites acting as a C&C for bots, worms, trojans, and other malware.
MEDIA SHARING
Sites that allow visitors to upload content and share media such as photos and videos.
MESSAGE BOARDS AND FORUMS
NEWS AND MEDIA
PARKED SITES AND DOMAINS
PROXY AND FILTER AVOIDANCE
REAL ESTATE
RELIGION
SEARCH ENGINES AND PORTALS
SHOPPING
SOCIAL NETWORKING
STREAMING MEDIA
SPORTS
TRANSLATION SITES
TRAVEL
UNCATEGORIZED
VEHICLES
VIRTUAL REALITY
WEAPONS
IAB1 – ARTS & ENTERTAINMENT
- IAB1-1 Books & Literature
- IAB1-2 Celebrity Fan/Gossip
- IAB1-3 Fine Art
- IAB1-4 Humor
- IAB1-5 Movies
- IAB1-6 Music & Audio
- IAB1-7 Television & Video
IAB2 – AUTOMOTIVE
- IAB2-1 Auto Parts
- IAB2-2 Auto Repair
- IAB2-3 Buying/Selling Cars
- IAB2-4 Car Culture
- IAB2-5 Certified Pre-Owned
- IAB2-6 Convertible
- IAB2-7 Coupe
- IAB2-8 Crossover
- IAB2-9 Diesel
- IAB2-10 Electric Vehicle
- IAB2-11 Hatchback
- IAB2-12 Hybrid
- IAB2-13 Luxury
- IAB2-14 Minivan
- IAB2-15 Motorcycles
- IAB2-16 Off-Road Vehicles
- IAB2-17 Performance Vehicles
- IAB2-18 Pickup
- IAB2-19 Road-Side Assistance
- IAB2-20 Sedan
- IAB2-21 Trucks & Accessories
- IAB2-22 Vintage Cars
- IAB2-23 Wagon
IAB3 – BUSINESS
- IAB3-1 Advertising
- IAB3-2 Agriculture
- IAB3-3 Biotech/Biomedical
- IAB3-4 Business Software
- IAB3-5 Construction
- IAB3-6 Forestry
- IAB3-7 Government
- IAB3-8 Green Solutions
- IAB3-9 Human Resources
- IAB3-10 Logistics
- IAB3-11 Marketing
- IAB3-12 Metals
IAB4 – CAREERS
- IAB4-1 Career Planning
- IAB4-2 College
- IAB4-3 Financial Aid
- IAB4-4 Job Fairs
- IAB4-5 Job Search
- IAB4-6 Resume Writing/Advice
- IAB4-7 Nursing
- IAB4-8 Scholarships
- IAB4-9 Telecommuting
- IAB4-10 Military
- IAB4-11 Career Advice
IAB5 – EDUCATION
- IAB5-1 7-12 Education
- IAB5-2 Adult Education
- IAB5-3 Art History
- IAB5-4 College Administration
- IAB5-5 College Life
- IAB5-6 Distance Learning
- IAB5-7 English as a 2nd Language
- IAB5-8 Language Learning
- IAB5-9 College / Graduate School
- IAB5-10 Homeschooling
- IAB5-11 Homework/Study Tips
- IAB5-12 K-6 Education
- IAB5-13 Private School
- IAB5-14 Special Education
- IAB5-15 Studying Business
IAB6 – FAMILY & PARENTING
- IAB6-1 Adoption
- IAB6-2 Babies & Toddlers
- IAB6-3 Daycare/Pre School
- IAB6-4 Family Internet
- IAB6-5 Parenting – K-6 Kids
- IAB6-6 Parenting teens
- IAB6-7 Pregnancy
- IAB6-8 Special Needs Kids
- IAB6-9 Eldercare
IAB7 – HEALTH & FITNESS
- IAB7-1 Exercise / Weight Loss
- IAB7-2 ADD
- IAB7-3 AIDS / HIV
- IAB7-4 Allergies
- IAB7-5 Alternative Medicine / Holistic Healing
- IAB7-6 Arthritis
- IAB7-7 Asthma
- IAB7-8 Autism / PDD / Asperger’s Syndrome
- IAB7-9 Bipolar Disorder
- IAB7-10 Brain Tumor
- IAB7-11 Cancer
- IAB7-12 Cholesterol
- IAB7-13 Chronic Fatigue Syndrome
- IAB7-14 Chronic Pain
- IAB7-15 Cold & Flu
- IAB7-16 Deafness
- IAB7-17 Dental Care
- IAB7-18 Depression
- IAB7-19 Dermatology
- IAB7-20 Diabetes
- IAB7-21 Epilepsy / Seizures
- IAB7-22 GERD / Acid Reflux
- IAB7-23 Headaches / Migraines / Fevers / Pain
- IAB7-24 Heart Disease / Stroke
- IAB7-25 Herbs for Health / Supplements / Vitamins
- IAB7-26 Holistic Healing
- IAB7-27 IBS / Crohn’s / Celiac Disease
- IAB7-28 Incest / Abuse Support
- IAB7-29 Incontinence
- IAB7-30 Infertility
- IAB7-31 Men’s Health
- IAB7-32 Nutrition
- IAB7-33 Orthopedics
- IAB7-34 Panic / Anxiety Disorders
- IAB7-35 Pediatrics
- IAB7-36 Physical Therapy
- IAB7-37 Psychology / Psychiatry / Therapy
- IAB7-38 Senior Health
- IAB7-39 Sexuality
- IAB7-40 Sleep Disorders
- IAB7-41 Smoking Cessation
- IAB7-42 Substance Abuse
- IAB7-43 Thyroid Disease / Endocrinology
- IAB7-44 Weight Loss
- IAB7-45 Women’s Health
- IAB7-WS1 Abortion
IAB8 – FOOD & DRINK
- IAB8-1 American Cuisine
- IAB8-2 Barbecues & Grilling
- IAB8-3 Cajun / Creole
- IAB8-4 Chinese Cuisine
- IAB8-5 Cocktails / Beer
- IAB8-6 Coffee / Tea
- IAB8-7 Cuisine-Specific
- IAB8-8 Desserts & Baking
- IAB8-9 Dining Out
- IAB8-10 Food Allergies
- IAB8-11 French Cuisine
- IAB8-12 Health / Low-Fat Cooking
- IAB8-13 Italian Cuisine
- IAB8-14 Japanese Cuisine
- IAB8-15 Mexican Cuisine
- IAB8-16 Vegan
- IAB8-17 Vegetarian
- IAB8-18 Wine
IAB9 – HOBBIES & INTERESTS
- IAB9-1 Art / Technology
- IAB9-2 Arts & Crafts
- IAB9-3 Beadwork
- IAB9-4 Bird-Watching
- IAB9-5 Board Games / Puzzles
- IAB9-6 Candle & Soap Making
- IAB9-7 Card Games
- IAB9-8 Chess
- IAB9-9 Cigars / Vaping / Tobacco & Accessories
- IAB9-10 Collecting
- IAB9-11 Comic Books / Anime / Manga
- IAB9-12 Drawing / Sketching
- IAB9-13 Freelance Writing / Getting Published
- IAB9-14 Genealogy
- IAB9-15 Getting Published
- IAB9-16 Guitar / Keyboard / Drums
- IAB9-17 Home Recording
- IAB9-18 Investors / Inventors / Patents / Copyright
- IAB9-19 Jewelry Making
- IAB9-20 Magic & Illusion
- IAB9-21 Needlework
- IAB9-22 Painting
- IAB9-23 Photography
- IAB9-24 Radio
- IAB9-25 Roleplaying Games
- IAB9-26 Sci-Fi & Fantasy
- IAB9-27 Scrapbooking
- IAB9-28 Screenwriting
- IAB9-29 Stamps & Coins
- IAB9-30 Video & Computer Games
- IAB9-31 Woodworking
- IAB9-WS1 Gambling
- IAB9-WS2 Weapons
IAB10 – HOME & GARDEN
- IAB10-1 Appliances
- IAB10-2 Entertaining
- IAB10-3 Environmental Safety
- IAB10-4 Gardening
- IAB10-5 Home Repair
- IAB10-6 Home Theater
- IAB10-7 Interior Decorating
- IAB10-8 Landscaping
- IAB10-9 Remodeling & Construction
IAB11 – LAW, GOVERNMENT, & POLITICS
- IAB11-1 Immigration
- IAB11-2 Legal Issues
- IAB11-3 Government Resources
- IAB11-4 Politics
- IAB11-5 Commentary
IAB12 – NEWS / WEATHER / INFORMATION
- IAB12-1 International News
- IAB12-2 National News
- IAB12-3 Local News
- IAB13-1 Beginning Investing
- IAB13-2 Credit / Debit & Loans
- IAB13-3 Financial News
- IAB13-4 Financial Planning
- IAB13-5 Hedge Fund
- IAB13-6 Insurance
- IAB13-7 Investing
- IAB13-8 Mutual Funds / ETFs
- IAB13-9 Options
- IAB13-10 Retirement Planning
- IAB13-11 Stocks
- IAB13-12 Tax Planning / Accounting
IAB14 – SOCIETY
- IAB14-1 Dating / Personals
- IAB14-2 Divorce Support
- IAB14-3 LGBTQ+
- IAB14-4 Marriage
- IAB14-5 Senior Living
- IAB14-6 Teens
- IAB14-7 Weddings
- IAB14-8 Ethnic Specific
- IAB14-WS1 Social Networking
IAB15 – SCIENCE
- IAB15-1 Astrology
- IAB15-2 Biology
- IAB15-3 Chemistry
- IAB15-4 Geology
- IAB15-5 Paranormal Phenomena
- IAB15-6 Physics
- IAB15-7 Space / Astronomy
- IAB15-8 Geography
- IAB15-9 Botany
- IAB15-10 Weather
IAB16 – PETS
- IAB16-1 Aquariums
- IAB16-2 Birds
- IAB16-3 Cats
- IAB16-4 Dogs
- IAB16-5 Large Animals
- IAB16-6 Reptiles
- IAB16-7 Veterinary Medicine
IAB17 – SPORTS
- IAB17-1 Auto Racing
- IAB17-2 Baseball / Softball
- IAB17-3 Bicycling
- IAB17-4 Bodybuilding
- IAB17-5 Boxing
- IAB17-6 Canoeing / Kayaking
- IAB17-7 Cheerleading
- IAB17-8 Climbing
- IAB17-9 Cricket
- IAB17-10 Figure Skating
- IAB17-11 Fly Fishing
- IAB17-12 American Football
- IAB17-13 Freshwater Fishing
- IAB17-14 Game & Fish
- IAB17-15 Golf
- IAB17-16 Horse Racing
- IAB17-17 Horses
- IAB17-18 Hunting / Shooting
- IAB17-19 Inline Skating
- IAB17-20 Martial Arts
- IAB17-21 Mountain Biking
- IAB17-22 NASCAR Racing
- IAB17-23 Olympics
- IAB17-24 Paintball
- IAB17-25 Power & Motorcycles
- IAB17-26 Basketball
- IAB17-27 Ice Hockey
- IAB17-28 Rodeo
- IAB17-29 Rugby
- IAB17-30 Running / Jogging
- IAB17-31 Sailing / Boating
- IAB17-32 Saltwater Fishing
- IAB17-33 Scuba Diving
- IAB17-34 Skateboarding
- IAB17-35 Skiing
- IAB17-36 Snowboarding
- IAB17-37 Surfing / Body-Boarding
- IAB17-38 Swimming
- IAB17-39 Table Tennis / Ping-Pong
- IAB17-40 Tennis
- IAB17-41 Volleyball
- IAB17-42 Walking
- IAB17-43 Waterski / Wakeboard
- IAB17-44 World Football / Soccer
IAB18 – STYLE & FASHION
- IAB18-1 Beauty
- IAB18-2 Body Art
- IAB18-3 Fashion
- IAB18-4 Jewelry
- IAB18-5 Clothing
- IAB18-6 Accessories
IAB19 – TECHNOLOGY & COMPUTING
- IAB19-1 3-D Graphics
- IAB19-2 Animation
- IAB19-3 Antivirus Software
- IAB19-4 C / C++
- IAB19-5 Cameras & Camcorders
- IAB19-6 Cell Phones
- IAB19-7 Computer Certification
- IAB19-8 Computer Networking
- IAB19-9 Computer Peripherals
- IAB19-10 Computer Reviews
- IAB19-11 Data Centers
- IAB19-12 Databases
- IAB19-13 Desktop Publishing
- IAB19-14 Desktop Video
- IAB19-15 Email / Chat / Messaging
- IAB19-16 Graphics Software
- IAB19-17 Home Video / DVD
- IAB19-18 Internet Technology
- IAB19-19 Java
- IAB19-20 JavaScript
- IAB19-21 Mac Support
- IAB19-22 MP3 / MIDI
- IAB19-23 Net Conferencing
- IAB19-24 Net for Beginners
- IAB19-25 Network Security
- IAB19-26 Palmtops / PDAs / Tablets
- IAB19-27 PC Support
- IAB19-28 Portable
- IAB19-29 Entertainment / Virtual & Augmented Reality
- IAB19-30 Shareware / Freeware / Open Source
- IAB19-31 Unix / Linux
- IAB19-32 Visual Basic
- IAB19-33 Web Clip Art
- IAB19-34 Web Design / HTML
- IAB19-35 Web Search
- IAB19-36 Windows
- IAB19-WS1 Hacking / Cracking
- IAB19-WS2 VPNs / Proxies & Filter Avoidance
- IAB19-WS3 Language Translation
- IAB19-WS4 File Sharing
IAB20 – TRAVEL
- IAB20-1 Adventure Travel
- IAB20-2 Africa
- IAB20-3 Air Travel
- IAB20-4 Australia & New Zealand
- IAB20-5 Bed & Breakfast
- IAB20-6 Budget Travel
- IAB20-7 Business Travel
- IAB20-8 By US Locale
- IAB20-9 Camping
- IAB20-10 Canada
- IAB20-11 Caribbean
- IAB20-12 Cruises
- IAB20-13 Eastern Europe
- IAB20-14 Europe
- IAB20-15 France
- IAB20-16 Greece
- IAB20-17 Honeymoons / Getaways
- IAB20-18 Hotels
- IAB20-19 Italy
- IAB20-20 Japan / China
- IAB20-21 Mexico & Central America
- IAB20-22 National Parks
- IAB20-23 South America
- IAB20-24 Spas
- IAB20-25 Theme Parks
- IAB20-26 Traveling with Kids
- IAB20-27 United Kingdom
IAB21 – REAL ESTATE
- IAB21-1 Apartments
- IAB21-2 Architects
- IAB21-3 Buying / Selling Homes
IAB22 – SHOPPING
- IAB22-1 Contests & Freebies
- IAB22-2 Couponing
IAB23 – RELIGION & SPIRITUALITY
- IAB23-1 Alternative Religions
- IAB23-2 Atheism / Agnosticism
- IAB23-3 Buddhism
- IAB23-4 Catholicism
- IAB23-5 Christianity
- IAB23-6 Hinduism
- IAB23-7 Islam
- IAB23-8 Judaism
- IAB23-9 Latter-Day Saints
- IAB23-10 Pagan / Wiccan
IAB24 – UNCATEGORIZED
IAB25 – NON-STANDARD CONTENT
- IAB25-1 Unmoderated UGC / Message Boards
- IAB25-2 Extreme Graphic / Explicit Violence
- IAB25-3 Adult Content
- IAB25-4 Profane Content
- IAB25-5 Hate Content
- IAB25-6 Under Construction
- IAB25-7 Incentivized
- IAB25-WS1 Content Server
- IAB25-WS2 Streaming Media
IAB26 – ILLEGAL CONTENT
- IAB26-1 Illegal Content
- IAB26-2 Warez
- IAB26-3 Spyware / Malware / Malicious
- IAB26-4 Copyright Infringement
- IAB26-WS1 Illegal Drugs & Paraphernalia
- IAB26-WS2 Deceptive / Phishing