Welcome, Guest!!
follow us on... rss

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - Mike

Pages: 1 ... 18 19 [20]
286
BotScout Discussion / Re: Just a note ....
« on: January 30, 2009, 10:06:46 AM »
Hi, I'm new here and have a problem. Today four times I try to get API but get an error
Help me please!

Welcome, Diabolic. Could you please post the user name, domain name, and email that you're trying to sign up with? If you don't want to post them publicly, please PM them to me.


287
BotScout Discussion / Re: Just a note ....
« on: January 30, 2009, 09:17:30 AM »
I've asked him to join over here, but just as a heads up;

http://www.stopforumspam.com/forum/p3053-Today-2%3A46#p3053
No problem; we'll be glad to help him out.

If you have any sample or development code that you'd like to share, we'd be more than happy to post it or link to it (and credit you, of course). 

288
BotScout Discussion / Re: Just a note ....
« on: January 30, 2009, 06:40:50 AM »
I'll definately need more than 300 per day (the SBST gets in excess of 5000 queries per day on a single site, and I've got it installed on 3 sites so far (no idea of the traffic to them on the other two sites as I don't monitor them)).
Wow, you weren't kidding....we had raised your initial query limit to 800/day, but I noticed first thing this morning that it had already topped out. I just raised it to 5000/day, and I'll keep an eye on it to make sure it doesn't get overrun again.

289
BotScout Discussion / Re: Just a note ....
« on: January 29, 2009, 11:06:03 PM »
Worked perfectly, cheers :)
Hmmm, I'm not certain, you might want to do it again. I'll have another look in the morning to make sure.

290
BotScout Discussion / Re: Just a note ....
« on: January 29, 2009, 10:03:57 PM »
Okay, let's try this again....delete the entry and then add it again. It should come up with the correct IP now.

291
BotScout Discussion / Re: Just a note ....
« on: January 29, 2009, 08:57:05 PM »
heh just added it, went to the URL you mentioned above (appended the form API key) and it's returning an "Invalid API key" error.

Where does it give me the file to upload to the server btw? (can't see an option for it)
Yep, it's not matching. We're looking at alternative ways to get a reliable IP address because the gethostbyname() function appears to have issues with some records.

The form (file) code isn't ready yet, but we expect to have a prelim version within a day or so. I'll get you a copy as soon as it's available.

292
BotScout Discussion / Re: Just a note ....
« on: January 29, 2009, 08:33:38 PM »
Weird. Hmmm. Well, go ahead and save it and I'll change the IP manually. Tell me which IP to put in there and we'll give it a shot.

The form server code is nearly done and should be available within a day or so. Once you get the form key you can go here to get a preview of the form server (the forms are visible now, but they won't be when it's released):

http://botscout.com/form/?apikey=your_form_api_key

(Note that the form key isn't the same as your regular API key)

We use the PHP function gethostbyname() to resolve the IP address, but apparently it's not reliable...so we'll need to find a reliable method of getting IP addresses. (Suggestions  welcome, lol)




It's detecting an IP now, but the wrong one ???

hosts-file.net = 212.56.95.253 (PTR: mysteryfcm.plus.com)

67.228.216.52 = eccparking.com

http://hosts-file.net/?s=67.228.216.52

Incidentally, I've got 3 sites on that IP listed in the hpHosts blacklist;

http://hosts-file.net/?s=67.228.216.52&sDM=1#matches

/edit

Using Avant Browser btw (uses the same Trident engine as IE)

293
BotScout Discussion / Re: Just a note ....
« on: January 29, 2009, 08:18:51 PM »
I've added this in and released the new version :)

Btw, the bot trapping form generator doesn't seem able to detect the domains IP? (domain is the hosts-file.net server)
I'll have a look at this, let me see what I can find. It should trigger once you TAB away from the field or click on the next field.

..........

Hmmmm, it seemed to find it okay for me: 67.228.216.52

Sometimes it takes it a moment or two to resolve the IP. If it still doesn't find it for you, let me know. Are you using IE or Firefox, or...?

I tried it with FF and IE 7.0 and it worked okay for me. (??)

294
BotScout Discussion / Re: Just a note ....
« on: January 29, 2009, 07:16:05 PM »
I sent you an e-mail earlier concerning the API key's usage, but decided what the heck, and figured I'd find out myself and it seems the key isn't site dependant, which is great.
Hmmm, did you send the email through the contact form? I looked but didn't see anything. (??)

295
BotScout Discussion / Re: Just a note ....
« on: January 29, 2009, 07:04:33 PM »
Nice one :)

Btw, your main site login form isn't allowing my full e-mail address in order to login :(

Full e-mail = botscout_com AT it-mate.co.uk

It seems to limit the chars so the last letter isn't allowed?
Whoops. That's fixed now. It should take up to 50 characters now.

My partner is adding the error handling modification to the API and says he'll be done by the time I finish this message. Here's the change to the API info page:

API Error Handling
In the event of an API error or query limit, the API will return an exclamation mark (!) followed by a single space, and then the text of the error message. For example:

! Sorry, but that doesn't appear to be a valid API key.

If your code tests for an exclamation mark as the first character and finds one, then it can expect an error message to follow. The message could be emailed to an admin. It's possible we'll add an error code for the various messages, but I'm not sure if there's much utility in doing that. (??)

296
BotScout Discussion / Re: Just a note ....
« on: January 29, 2009, 06:54:33 PM »
Much appreciated, cheers :)

A return value specific to errors would be much appreciated as it would allow easier differentiation from non-matches (let me know when you've done this and I'll modify the scripts accordingly).
We'll look at adding this within the next day or so and we'll post a notice here (the API Doc page will also be updated).

297
BotScout Discussion / Re: API Query Discussion & Info
« on: January 29, 2009, 06:50:25 PM »
Additional Note:

Testing an email or IP address all by itself will give you a very good likelihood of detection with a very, very low false positive rate (well under 1% according to my partner). It's what we recommend for most users.

Testing a second item really doesn't buy you that much, in fact in some cases it actually confuses the issue. It's like having two watches that disagree- which one is right? Let's say the IP matches but the email doesn't (or vice versa)...now what do you do?

298
BotScout Discussion / Re: Just a note ....
« on: January 29, 2009, 06:45:20 PM »
hehe cheers :) ....... I'll definately need more than 300 per day (the SBST gets in excess of 5000 queries per day on a single site, and I've got it installed on 3 sites so far (no idea of the traffic to them on the other two sites as I don't monitor them)).
Yikes, lol. Okay, we'll keep an eye on the query count for your key and if it starts to climb we'll bump it up (probably way up).

The one bit of advice I'd offer in terms of implementing the code is to check the return values carefully, the absence of a 'Y' doesn't necessarily mean there was no match- it may mean that an error is being returned (bad format, query limits, etc). We're looking at adding a special return value to help implement this, most likely a "!" returned as the first character to give scripts something to trigger on.

299
BotScout Discussion / API Query Discussion & Info
« on: January 29, 2009, 06:41:24 PM »
Original article: http://botscout.com/api_queries.htm

API Query Info

This page discusses some of the considerations involved in using the different kinds of queries available in the BotScout API.

    * The "MAIL" Query

      The MAIL query is one of the simplest and most useful query types in the BotScout API. The MAIL query takes an email address and looks for matches in the BotScout database, searching only in the email field. The email address is, statistically, the best indicator of whether or not a potential "user" is in fact, a bot. This is because virtually all forums and other web services now require that a valid email be used when signing up or registering to gain "post" access. Post access is the level of user access required to (for example) leave a message on a message board, fill out a classified ad, or enter data into a database.

      The valid email address is used to confirm receipt of the registration credentials so that bots can't flood a site with thousands (or tens of thousands) of fake user accounts. Since a working email address is required, this provides a credible pointer to a given user (or bot). Although it's common for a single bot to make use of many different email addresses, the available addresses are often used and re-used. Email addresses used by a bot are almost never used by an actual human (a "real" user) so the chance of a collision or false positive is very, very small.

    * The "IP" Query

      The IP query takes an IP address and looks for matches in the BotScout database, searching only in the IP address field. IP addresses by themselves are reasonably good indicators of bot activity, but since many bots operate from computers that have been infected by malware and joined into a botnet, the chance that a given IP address will correspond to a "real" user's IP address is possible (although very low in practical terms).

      To get an idea of the collision rate for a given IP address, consider the following:

      There are 4,294,967,296 (4.3 billion) possible IP addresses, of which 2,147,483,648 (2.1 billion) are normally available to typical service-level users (and bots).

      If the BotScout database had 100,000,000 (100 million) stored IPs and there were 100,000 new users that tried to sign up to a BotScout-screened service tomorrow, the odds of a collision would still be incredibly small. 100 million IPs is one-quarter of one percent (.025%) of the available IP addresses. 100,000 is one one-thousandth of 100 million. In simple terms, the chance that a given IP will be shared by a bot AND by a user who wants to sign up to your forum or web service is, for all practical terms, incredibly small.

    * The "NAME" Query

      The NAME query takes a given user name and looks for matches in the BotScout database, searching only in the NAME field. By itself, the NAME query is much less reliable than an IP or MAIL quey and should not be relied upon for bot screening. It should be used ONLY as a secondary indicator, and even that is of dubious value. Name collisions are not uncommon since bots use nonsense names as well as "real" names when they run. The NAME query by itself is next to useless; if used it should always be coupled with a MAIL or IP query for reliability. The false positive rate of using the NAME query alone is abysmally high.
    * The "ALL" Query

      The ALL query takes a given data item (IP, NAME, or MAIL) and looks for matches in the BotScout database, searching against all of the database fields. This query has limited usefullness, but was provided primarily because some bots will use an email address as the user name. Sometimes this is by design, sometimes this is likely due to operator error or misconfiguration of the bot's operating parameters.

    * The "MULTI" Query

      The MULTI query is specialized type of query that, when used correctly, can provide a very high level of detection with a very low false positive rate. It requires more involved parsing on the requester side, and (optionally) some decision making capability built into the processing code. It is the recommended default for querying the BotScout database because it returns the most usable data for a single query.

      The MULTI query takes all three data items (an IP address, a name, and an email address) and looks for matches in the BotScout database, searching against all of the database fields uniquely. That is, names will be compared to the NAME fields, IPs will be compared to the IP fields, and the email address will be compared to the MAIL fields.

      A composite set of matches with occurrence numbers is returned for all of the items, whether they matched or not. If, for example, the IP and the email matched, they will have numbers showing the times each item was found in the database. The name field would also be returned, but would show zero matches.

      The MULTI query can be used to give a reasonable statistical certainty as to whether the items submitted constitute bot activity or not. The phpBB plugin available for BotScout makes use of the MULTI query.

Summary
In general, Email is the single most reliable indicator of a bot- 99.9% or better. The IP address is the second most reliable indicator of a bot- 95% or better. The name is the least reliable indicator of a bot. It varies too much to be assigned a real percentage- possibly 30%.

Because bots routinely change their IPs, names, and emails, there's a very good chance that testing for a unique 3-item combination won't return a match and the registration would go through- which is exactly what we don't want. In fact, you're radically reducing the likelihood of catching them for each additional item you test for.

If you tested for any one of those items you stand a very good chance of catching them. Testing for two items drops that to ~25% or so, based on all of the unique bot signatures in our database so far (almost 95,000). Test for a positive match on all three items causes the probability to drop way, way down. A match on three items would be a guaranteed bot, but you'd be very lucky if you nailed them the first time through...and all they need is one successful registration.


300
BotScout Discussion / Re: Just a note ....
« on: January 29, 2009, 06:35:58 PM »
I sent you an e-mail earlier concerning the API key's usage, but decided what the heck, and figured I'd find out myself and it seems the key isn't site dependant, which is great.

I've went ahead and included a BotScout lookup in my Spambot Search Tool :) (I use this on multiple sites for multiple forums/forms/blogs/etc)

Welcome, and thank you for implementing the API in your software!

You're correct, the API key is not site-dependent; you can use the same API key on as many sites as you need to. Currently the key is good for 300 API queries a day, but we can adjust that up if you need more (just let us know).

We're almost ready to release our Bot Trap Form Server, and you're more than welcome to use that as well. We're putting the finishing touches on it and hope to have it available within a few days.

The Form Server lets people put a small line of code in a page that invisibly embeds a randomized form in the page each time it's loaded. The form is very attractive to bots and is designed to be nearly impossible for them to identify as a bot trap.

Unfortunately, the Form Server system needs a different key for each site registered; this was a design consideration forced on us in order to prevent malicious users (botnet owners) from poisoning the database. We don't have a hard limit on the number of sites allowed yet, and the limit will be adjustable on a per-user basis, so trusted users will be able to deploy more forms than other users.

The system uses a scoring ladder to help weed out false entries, and should be very effective in proactively responding to bots and letting them add themselves to the database. :)

If you're interested in trying out the Bot Trap Form Server, let me know and we'll get you going. It's very easy to implement.

Again, welcome and thank you for implementing the BotScout API in your code!


Mike

Pages: 1 ... 18 19 [20]