Possible solution to carrier blocking

[vdrey]

So in #11, people were saying that they think that Tmobile is blocking textbelt.com.

We don't know if they are doing that, but if they are, it is likely because they were receiving massive amounts of emails from the textbelt.com domain, due to textbelt not being able to determine what number corresponds to what carrier, and sending messages to all carriers on the list. I think I might have a solution to that.

Before I describe that solution, just know: I don't know anything about JS, so I am of no use in implementing this solution, or determining whether it is feasible or not. This is just an idea I had.

As far as I understand, when an email is sent using smtp, and the user does not exist, the server responds with error code 550.

My idea is that if textbelt tracked which servers responded with 550, and which server accepted the message, you could determine the carrier for that number, and only send messages to that carrier in the future.

Again, I could be completely wrong, but this is just what came to mind. Any thoughts?

[typpo]

That would certainly help. But then you run into edge cases like what if the user changes their carrier, and you have it cached?

Another way to mitigate the problem would be to allow the user to specify their carrier in the API. Repeat users might supply that info to increase likelihood of delivery.

Other workarounds are to set up another domain (if they banned the domain) or another server (if they banned the mail server), and then aggressively reject spammy texts so we don't get banned again.

Ultimately, the best way for individual power users is to self-host Textbelt (TMobile has only banned my textbelt.com, not you) or pay for better delivery guarantees via Twilio or similar.

I appreciate your thoughts on all this!

[vdrey]

If a user changes their carrier, the old carrier should respond with 550, and textbelt could re-send the message to all carriers, and determine the carrier from there.

Also, if they switch to a carrier not supported by textbelt, all carriers would respond to 550.

[daluu]

Interesting post, good to know. I would +1 for having option to specify carrier in the API, with default being unspecified. If user chose to mention carrier we can then avoid dealing with this case, and only deal with it if needed (default case). The option reduces the frequency of the need to maintain a valid lookup cache of SMS email gateway servers per number.

This other free public SMS API takes the approach of specifying the carrier (required) on purpose: http://www.watacrackaz.com/autosms/api.php. It's a viable alternative to TextBelt IF they offered other restriction/limiting methods than enforcing CAPTCHA. I believe that one works for T-Mobile.

Also this lookup cache solution is good, but increases storage in the system in order to maintain the lookup cache as you scale up. Although it may be negligible as it's short text table mappings, but something to keep in mind based on user base size. It may be less of an issue for a private hosted TextBelt scenario where you have more control and know your user base. For the public service, it's really based on how popular the service is and gets in the future.

Another case to consider for this solution is how often do we purge that lookup cache? I guess it could be made configurable (for self hosted option) but it will be some default value for the public service. The cache could get really big over time if it's never purged.

Lastly, this solution is only effective for repeat users (e.g. user sending a message to same number again), reducing amount of blast emails to all the carriers over time. But for a new user or a user sending to a new number, we start all over again since the number is not in the cache. And if we purged cache periodically, we start all over more frequently since the number that was cached may be gone now, for the sake of saving storage (space/cost). So this proposed solution is a mitigation factor but not a complete solution to the core issue.

[vdrey]

Yeah, my "solution" is really just a mitigation, but I think it, along allowing users to specify the carrier, would be pretty powerful.

As far as purging the cache goes, I have 2 ideas.

The first is that records are removed on a rolling basis after a certain amount of time. In that situation, after a set amount of time, a record would be purged.

The other option would be to set a certain maximum cache size, and when a new record becomes available once that maximum size limit is hit, the oldest record in the cache is deleted.

Again, these aren't permanent solutions, but they seem like they could help reduce the volume of emails sent by textbelt, and hopefully prevent carrier blocking.

[ksparakis]

Can definitely confirm T-mobile, seems to be blocking TextBelt.

[scotthurlbert]

I'm also beginning to suspect that Verizon has blocked TextBelt as well. It was working until 1:30PST on 10/29/2015, but I haven't received any messages in the past 24 hours. I just sent myself several messages that never arrived despite [success=true].

[RossCreager]

Tested Verizon and ATT. Both did not work even though I received a success response.

[daluu]

Last time I tried over the weekend, it seemed that the text didn't go through on Sprint either. It used to work. I wonder if the carriers are all getting tough on textbelt, or if textbelt has started getting a lot more usage such that it would lead to the carriers to get tougher.

[typpo]

Very puzzled by this. I think there might be something wrong with the server sendmail config, but I haven't changed anything recently. The messages appear to be delivered for most providers.

When I construct the sendmail command manually, it says delivered but no text. If I send through other means (eg. mutt), I get a text.

This means it's likely something wrong with the way sendmail is set up, because if textbelt.com was banned I wouldn't be able to get it through by other means.

Wondering if anyone here self-hosts and whether things are still working for you?

[typpo]

I think I've restored functionality for providers like Verizon and AT&T that stopped working last week. This probably doesn't fix the TMobile problem because they seem to have blocked textbelt.com.

[danry25]

T-Mobile seems to work just fine for me thru textbelt, just sent myself 5 texts before hitting the quota restrictions.

[wa0uwh]

Has there been any progress on the problem resolution? Verizon has stopped working.

For a suggested solution: How about requiring each user to "seed" a data set with the a phone number's carrier before it can be used. This would avoid thousands of errant connections to the wrong carriers SMS API, and therefore avoiding the potential Black Listing. The seeds could be set to "self expire" after a time, and a seed could be "reset" by a user, if a user changes carrier.

Also, multiple Domains could be used by Textbelt, where seeded number would be sent from a trusted domain name, while un-seeded numbers would be "attempted" by another. If the un-seeded is found to be successfully transferred, it could be self-seeded (with maybe a different seed expatriation configuration) and therefore it would be candidate to be sent (the next time) via a semi-trusted domain name.

I think 95% of legitimate uses will know the phone number's carrier, seeding would be a small inconvenience for a usable and stable SMS API service for a few phone numbers, used by a typical user.

Also, there are many suggested alternative services to Textbelt, but most (if not all) require a "clear text" password to be inserted into a configuration file (a very BAD idea). I do not think most people understand the usefulness of a unauthenticated SMS service which has a quick and easy SMS's API. This is especially true as the "Internet of Things" (IoT) devices grow in numbers. For the IoT's a small foot print SMS API is very useful.

I would hate to loose this service for lack of not finding a Phone Carrier acceptable and workable solution.

So, any progress?

Thanks,
Eldon

[daluu]

If I'm not mistaken, this item is not being (actively) worked on, not a priority. Those who feel this compelling, vote it up. And for further action, you (or find someone) to implement the solution (in a fork) and issue a pull request to get it done promptly.

[typpo]

Can you guys try hitting the endpoint at txtblt.co instead of textbelt.com? eg. curl -X POST http://txtblt.co/text -d number=2121234567 -d "message=foo bar testing"

I've been playing around with various fixes and am able to get a text through on my provider (Verizon). I'm still not sure if the problem is on my side or if carriers are actively trying to block.

Eldon, I agree that it would be great to have a way for users to specify the carrier. I will make this a priority but I'm not sure I can provide an ETA given the holidays etc.

[wa0uwh]

That worked for me. I received a text message from "[email protected]" on my Verizon phone near Seattle.

I would suggest, if you are considering allowing the user to "seed" the carrier name, required it to be included as part of the phone number for all first time use, as "[email protected]". Personally, IMHO, I think that if a user does not know the carrier, then they are fishing.

Thanks, typpo, I appreciate your efforts.
Eldon

[typpo]

Great. I will try to confirm a few other providers via people I know. If the other providers work, I will point textbelt.com to the txtblt.co server and see if things still work.

As an aside, all of you are free to self-host the textbelt node app and send from any domain you prefer :)

[wa0uwh]

I just install the new code into one of my IoT (an Esp8266) web servers, it works great.

As an example: My IoT is not always OPEN and available on the Internet, but for a test and a while Textbelt can be seen in use, at: http://dc02.ebcon.com:8129/admin .

Also, see: https://hackaday.com/2015/09/05/esp8266-web-server-farm

Regards,
Eldon

[typpo]

I redirected textbelt.com to the new server. Anecdotally (read: my Verizon phone + my friends with AT&T and TMobile) people are now receiving texts.

I will merge the carriers pull request and aggressively respond to spam to prevent future blacklisting, hopefully.

[daluu]

@typpo, "aggressively respond to spam"? What do you mean by that?

[typpo]

There are a handful of users who run spam campaigns via Textbelt. I have banned a few but I don't spend enough time tracking these. I'll probably add more monitoring and continue to ban them.

[peeeeeter]

A couple years ago, I had the same idea that the original post was about. I run a hosting company, and from time to time would run a manual telnet SMTP session, one command at a time. In the back and forth of an SMTP session, you specify the "To" address, then get back a response, which SHOULD include and error if the "to" address doesn't exist. My initial testing showed I could get half way through the SMTP session, then abort if I get the error after specifying the to address, thus avoiding a bounce or ban.... I wrote it all up in php or perl, staying up half the night. Once I finished and plugged in more carriers, I realized, MUCH to my disappointment, that half of the US carriers DID NOT check for a valid "TO" address. So, I chalked that up as a waste of time. I guess it's possible more of the carrier's gateways do that now.

I just wanted to share my experience with the original poster's idea..

[sam-nuntio]

Typpo thanks for all the work on textbelt so far. I am in dire need for such a solution to text me when our startup's experiments are failing so we wake up in middle of the night to fix it instead of waiting till next morning. Although in theory textbelt would have been perfect fit but unfortunately texting to my number or my co-founder's number (both AT&T) did not deliver the message even after receiving success. Any known issues or any way to ensure delivery ?

[karoltheguy]

It has stopped working for me as well here in Canada with Bell Mobility.