• Top
  • New

Show HN: Check if a Domain is common or sketchy

by Iwillgetby on 6/18/2020, 9:01:03 PM with 1 comments
PS C:\> Invoke-Sqlcmd -Query "sp_select_from_domains_where_domain_starts_with 'amIevil';" -ServerInstance " infosecrockstars.eastus2.cloudapp.azure.com ,443" -username "readonlyuser" -password "twitter"

rank, domain

---- ------

28193044, amievil-graphicnovel.com

2611848, amievil.com

37416981, amievil.myblog.de

143664338, amievillegas716.joomla.com

160676189, amievilmovie.com

196148720, amievilt.com

PS C:\> Invoke-Sqlcmd -Query "sp_select_from_domains_where_domain_equals 'cnn.com';" -ServerInstance "infosecrockstars.eastus2.cloud app.azure.com,443" -username "readonlyuser" -password "twitter"

rank, domain

---- ------

66, cnn.com

These 2 stored procedures query a table with a Billion domains in it.

  • by Iwillgetby on 6/18/2020, 9:04:31 PM

    This is the original data source.

    harmonic centrality / pagerank

    https://commoncrawl.org/connect/blog/

    12.16 GB

    cc-main-2020-feb-mar-may-host-ranks.txt.gz