Why do this at all?

Yes, this is a post about sending faxes. It’s a technology that should be dead in 2021, but isn’t because old technology dies hard. Want to send some paperwork and they don’t have a website form, but don’t want to overnight it? You’re probably sending a fax. Want to get a digital copy of a document from somewhere that doesn’t send digital copies of things, but has been around for a while? You’re probably going to receive a fax.

For many years, I used Faxage for this purpose. They’re a good company and I never had any trouble with them, but they charge $3.49/mo and you pay five cents a minute on top on the minimum plan. They are perfectly good, but $40/year plus usage adds up over time.

I thought that I’d found the answer when Twilio introduced fax support. The pricing was set at the same number for their programmable voice, which is to say a dollar a month for the number, and then a cent a page for receiving or standard per-minute pricing for sending.

I had to write some software to handle integrating with the new stuff, but it wasn’t too bad, and I got it working with their API in late 2018. It was still a little buggy then, but by early 2020 it was more reliable and I decided to port my number out of Faxage and close the account.

At the end of 2020, Twilio announced that they would be shutting down programmable faxing altogether come December 2021. I wasn’t super thrilled by this, but settled on figuring out a migration path in the next year or so. I could obviously go back to Faxage, but Twilio has great pricing. Twilio does have a migration path to a different company, but I didn’t want to have to change how all my cloud faxing stuff worked to pay more.

I finally decided I’d just try to set something up at home. It would be more work, but I probably wont have to change it for some time. It’s also easy to swap out VoIP providers if I want to do that in the future.

Setting up Faxing at home

For a while, my home network connection wasn’t the most reliable, and I’d moved a number of things to cloud service providers versus running things there. I haven’t bothered with having a phone line in many years, especially since it’s so cheap to use wireless services these days. Even if you can order a traditional twisted pair copper phone line from your local telephone company (many aren’t even offering it these days), it doesn’t come cheap because the infrastructure is aging. Even if it’s running over a modern network, you’re still probably going to pay something like $20/mo for something you might not use often.

I’m lucky to currently have fiber to the home, and that stuff is rock solid. A reliable network is a good candidate for running fax using the standard T.30 faxing protocol and standard G.711 audio codecs. In other words, old school fax machines going over what looks like an ordinary telephone line, only using the internet.

There’s a lot of old information out there about how you can’t do this very thing. For example, I’ve read some things that suggest that a home network is going to have too much jitter (variability) in the audio packets, which can cause data loss on the packet. Other things I’ve read suggest that compression will mess up things, which is true if you are using low-quality VoIP trunks that will recompress the audio.

I’m happy to say in my testing, these don’t seem to be actual problems.

How do you get a home phone line using Twilio?

Let’s start off with the phone line part. Twilio will let you set up a SIP domain using their standard programmable voice stuff. You can use SIP trunking, but I found that the latter was less complicated for me coming from scripting Twilio already.

In years past, perhaps 15 years ago, I used to use Sipura Phone Analog Telephone Adapters (ATAs) for this purpose, such as the SPA-1001. It worked perfectly fine, but it’s aging hardware at this point. Cisco/Linksys bought them in 2005, and they rebranded over to Cisco in 2009. As it turns out, Cisco still sells this gear, and some of it is still supported.

Looking on eBay, I found a relatively recent one (SPA122) for around US $30 with shipping used, but said to be working. When I plugged it in, I found that I couldn’t get to the admin screen to set it up fully and the non-admin user information said that it was locked to Big River Telephone. I did manage to get the thing unlocked so that I could configure it, but that is a separate post.

What about the fax machine itself?

There is no shortage of Fax machines on eBay. I saw that I could get one for as little as $40 that used regular paper and a thermal transfer roll. A cheap fax machine would get the job done for scanning documents in standard paper formats, and printing stuff out, but there’s a few problems with that. For one, you have to be home to send or receive a fax. Printing stuff is kind of wasteful too, and some of the technologies use thermal transfers that rely upon BPA, and no need to have more of that kicking around nearby.

I decided that the best way to handle this would be to get a fax modem and hook it up to a computer. I did consider at this point going for a pure software solution, e.g. IAXmodem. I’ve set up Asterisk in the past and could use that to provide trunking for IAXModem, but that would add more layers of complexity over using an analog telephone adapter and be less tested.

When I went to look for a fax modem, I learned that US Robotics, the company that was known for making modems in the 1980s and 1990s is somehow still around. While USR got bought by 3Com in 1995, 3Com decided to spin them back out in 2000 and there’s enough of a modem business that they are still around. In around 2010 they came out with a 56K USB Modem with Class 1 Fax support, and which works with Linux. I bought one of these on eBay (USR5637) for around US $15.

What about the fax machine software?

There are two types of fax modems.

There’s the Class 1 fax modem type, where the computer essentially does all the processing. This is good for updates to the protocol and fixing bugs because some of this stuff is never really done. On the other hand, it means that the computer has to do more work, which I’m only mentioning because this was a big deal in the early 1990s when these things came out, and a lot of the documentation for fax-related stuff is fairly old. It’s clearly the right choice for getting updates when the technology has been essentially fixed for several decades.

There’s also the Class 2 fax modem type where the protocol is baked into the modem, and the computer has to do less work. Given that computing has advanced a lot, this is essentially just getting stuck in the past for really no gain unless you plan to write your own faxing software from scratch or use archaic software.

Since faxing hasn’t been a hot technology for quite some while (even in Japan where it’s popular), there aren’t a ton of choices for faxing software, but HylaFAX and it’s fork, HylaFax+ are two decent ones. I opted to use the non-forked version because it was less effort to set up because it’s still distributed using Debian. If you’re interested in the differences, and care enough to build from source, you can see the differences on the HylaFax website.

So how do I configure this thing?

I think the documentation for this stuff is pretty bad, so let me try to summarize how this stuff works in a nutshell.

You’re going to need a modem to work with this software, which you’re going to talk to over a device node. If you’re talking to an external modem over pure serial, this is going to be wherever your serial port is, such as /dev/ttyS0 or /dev/ttyUSB0. For something like my USR 5637 modem, it’s at /dev/ttyACM1, because it’s the second ACM USB device I have on this particular system, but it might be at /dev/ttyACM0 for you if you don’t have anything else on there.

You’ll need something that can coordinate access to that device node, which is typically some getty clone with special capabilities. Since I’m going to be using a Class 1 modem, this is going to be HylaFAX using faxgetty.

To install the Hylafax on Ubuntu 20.04 LTS at least, it’s as straightforward as apt install hylafax-server hylafax-client. I forget if it automatically will try to set up the modem for you, but if it doesn’t, you can use faxaddmodem to get it to do some auto-configuration. This will make some guesses about how to set up the modem, many of which are wrong for the USR5637. In particular, automatic answering and flow control will be wrong, and this will prevent you from sending and receiving faxes.

Thankfully, someone in the last few years someone on a German-language Raspberry Pi forum did the hard work of figuring out how to set it up.

Over in /etc/hylafax/config.ttyACM1 or wherever your modem lives, you’ll want to edit the modem-related configuration to include this:

# Modem-related stuff: should reflect modem command interface
# and hardware connection/cabling (e.g. flow control).
#
ModemType: Class1 # use class 1 interface
ModemRate: 19200 # rate for DCE-DTE communication
ModemFlowControl: rtscts # software flow control
#
ModemSetupDTRCmd: ATS13=1&D3 # setup so DTR drop resets modem
ModemSetupDCDCmd: AT&C1 # setup so DCD reflects carrier (or not)
ModemNoFlowCmd: AT&H0 # setup modem for no flow control
ModemHardFlowCmd: AT&H1 # setup modem for hardware flow control
ModemSoftFlowCmd: AT&H2 # setup modem for software flow control
ModemResultCodesCmd: ATQ0X4 # enable result codes
#
ModemMfrQueryCmd: !USR5637
ModemModelQueryCmd: ATI3
ModemRevQueryCmd: ATI7 # XXX returns a multi-line result
#
# When AT+FCLASS=1 is issued the modem automatically switches
# to software flow control; these parameters let the fax software
# reset flow control as needed after entering Class 1.
#
Class1NFLOCmd: AT+FLO=0 # setup modem for no flow control
Class1HFLOCmd: AT+FLO=2 # setup modem for hardware flow control
Class1SFLOCmd: AT+FLO=1 # modem does this automatically
#
# This should resolve "DIS/DTC received 3 times" errors:
#
Class1ResponseWaitCmd: AT+FRS=1 # wait after sending TCF for response
#
# The remainder of this configuration is included so that the
# modem "idles" in Class 0 while not sending or receiving facsimile.
#
#ModemSetupAACmd: AT+FCLASS=0 # leave modem idling in class 0
#ModemAnswerCmd: AT+FCLASS=1;A # answer in Class 1
#
# When using AT+FRS=n we see USR modems reset themselves in the middle of sessions
# this is not good.  So, we seem to work-around that problem by not using the
# command.  Unfortunately, this isn't an ideal thing.
#
Class1SwitchingCmd: "<delay\0727>"

Once that’s there, go ahead and do a systemctl restart hylafax to restart faxgetty and friends.

Also, if you have more than one ACM device and want the configuration to be stable, then something like /etc/udev/rules.d/99-faxmodem.rules with something like this will make a /dev/faxmodem symlink for you (try udevadm info --attribute-walk --name /dev/ttyACM1 or the like if you have a different model modem):

SUBSYSTEMS=="usb" DRIVERS=="usb" ATTRS{idVendor}=="0baf" ATTRS{idProduct}=="0303" SYMLINK+="faxmodem"

If you have that set up, you can rename your configuration to something like /etc/hylafax/config.faxmodem.

Okay, it’s configured, how do I send and receive faxes?

You send a fax with the helpful sendfax command. The command helpfully gives you no cli help options and tells you to just read the man page. By default, it will fill out a cover page for you that you can add stuff onto, and will take stuff in plain text, TIFF, PostScript and PDF. I mostly use PDF for this purpose without a cover page, but it’s convenient that it will make a coverpage.

Once you’re set up, you can do this with something like:

# -n for no cover page
# -d for the phone number
sendfax -n <phone-number> <mydocument.pdf>

You can read the man page if you want to use the cover page, which appears to be very customizable.

Here’s an example: sendfax -G -n +15555555555 foo.pdf. Note that I have all my ATA stuff set up to automatically handle ‘011’ turning into a ‘+’ for E.164 dialing. You may set up your dialing plan differently, and HylaFax has some options for it. The -n argument says “no cover page”, and the -G argument tries to negotiate the best quality possible.

There are some other useful arguments for sendfax, such as -B 9600 and -s a4 to reduce the speed and send in A4 page size, if you’re faxing from the US internationally.

You can check on the progress for sending (and receiving) with faxstat -r. If it says that it is wedged or waiting for the modem to come ready, it’s probably broken, and you’ll want to read the logs.

If you think the thing on the top of the fax looks wrong, that’s called the TagLineFormat and will require some tweaking. The configuration for it is found in the hylafax-config man page. It sits in the same file as the modem configuration.

The LocalIdentifier is where you’d put your name, and probably your return fax number. At least put a return fax number there. The middle part will be a %c by default and is the output of strftime formatting. You can and play with this format, with date [+output_fmt]. The default will display as date +%c, but I personally opted for a modified ISO8601 (%F %T %Z) to be the most accessible.

For receiving faxes, the modem should automatically answer and faxgetty will process the fax. You can set it up to send you a mail when this happens, but in the most basic form, it will dump files into /var/spool/hylafax/recvq as .tif files using the seqf digit to figure out what number to append to the file.

If you want to get an e-mail when you receive a fax, you’ll want to tweak the /etc/hylafax/FaxDispatch configuration. Something like:

FILETYPE=pdf;
SENDTO=<your-email@example.com>;

should do the trick, though e-mail obviously needs to be functional on your system already.

Also, if you want to get an e-mail confirming that a fax was successfully sent, then add something like this to your hyla.conf:

Notify: done

There are scripts for tweaking behavior around send/receive in /var/spool/hylafax/bin, such as faxrcvd for inbound faxes and notify for outbound faxes, but I haven’t yet tweaked them. These are a good starting point if you want to make your own custom integration.

Final Thoughts

If you’re on this page, you’re probably annoyed like I was trying to get faxing to work, but hopefully this gives you the answers you need. I also now have a migration path for the Twilio faxing API going away, and a migration path if I need to switch VoIP providers in the future.

If you’re in the same boat for that deprecation, I’d suggest getting something working before December 2021 because it will be harder to test a new setup later.