pfSense using urlAlias local filtering

Leave a comment

November 4, 2016 by aubreykloppers

The steps here are quite complicated and a couple of things need to be done depending on which platform of pfSense you are.  To find Facebook IP’s/Networks and successfully block/shape Facebook (and now Microsoft and Youtube) is quite a tricky thing, but if you know how to, it becomes managable.

pfSense 2.4+:

You will need to install BASH to run the script.  To do this run pkg install bash (on 2.2+ you will also have to install whois with pkg install whois) and it will be installed into the /usr/local/bin/bash directory.  The commands you will use here  will be:

  • Facebook:
whois -h whois.radb.net -- \
"-i origin AS32934" | awk "/^route:/ {print $2;}" | sort | uniq
  • Microsoft:
whois -h whois.radb.net -- \
"-i origin AS8075" | awk "/^route:/ {print $2;}" | sort | uniq

So, now that you have all the tools in your hands to create your listings, onto the alias creation:

Alias Creation:

  • Create a folder in /usr/local/www called aliastables:
cd /usr/local/www
mkdir aliastables
cd aliastables
  • Create your Facebook and Microsoft aliases.  Remember, you can run a cron-job to re-create this list:
whois -h whois.radb.net -- "-i origin AS32934" | \
awk "/^route:/ {print $2;}" | awk '{print $2}' | \
sort | uniq > facebook.txt

AND

whois -h whois.radb.net -- "-i origin AS8075" | \
awk "/^route:/ {print $2;}" | awk '{print $2}' | \
sort | uniq > microsoft.txt

This will create two files, namely “facebook” and “microsoft”.  You can use these two files now to address via pfSense by going to:

  • Firewall:Alias and Select “URL Table”.  Enter the URL as:
http{s}://{pfsense-ip}:{port}/aliastables/{file}
i.e:
http://127.0.0.1/aliastables/facebook.txt

Troubleshooting:

I had some problems with the creation of local URL tables, but found using an off-site FQDN will gets the job done.  I have logged the problem on the pfSense forum and hope to get clarity on this soonest.  (see this pfSense Discussion)

Fixing the problem:

If you get the error (I will give you Youtube, Facebook and Microsoft AS for a full range of filters):
Unable to fetch usable data from URL http://127.0.0.1/aliastables/youtube/microsoft/facebook.txt
you will unfortunately have to create an alias (any alias will do, just use it as a marker), then edit /cf/conf/config.xml and replace the alias with the following:

<alias>
  <name>facebook</name>
  <type>urltable</type>
  <url>http://127.0.0.1/aliastables/facebook.txt</url>
  <updatefreq>128</updatefreq>
  <address>http://127.0.0.1/aliastables/facebook.txt</address>
  <descr><![CDATA[Facebook AS List.]]></descr>
  <detail><![CDATA[Facebook AS List.]]></detail>
</alias>
<alias>
  <name>microsoft</name>
  <type>urltable</type>
  <url>http://127.0.0.1/aliastables/microsoft.txt</url>
  <updatefreq>128</updatefreq>
  <address>http://127.0.0.1/aliastables/microsoft.txt</address>
  <descr><![CDATA[Microsoft AS List.]]></descr>
  <detail><![CDATA[Microsoft AS List.]]></detail>
</alias>
<alias>
  <name>youtube</name>
  <type>urltable</type>
  <url>http://127.0.0.1/aliastables/youtube.txt</url>
  <updatefreq>128</updatefreq>
  <address>http://127.0.0.1/aliastables/youtube.txt</address>
  <descr><![CDATA[Youtube AS List.]]></descr>
  <detail><![CDATA[Youtube AS List.]]></detail>
</alias>

You can now create a script to run the replacement of the firewall rules at startup to overwrite the files and load the content into the filter.  This is done by adding a command into “Services/Shellcmd“, creating the command “/script/aliases.sh” and change the “Shellcmd Type” to “afterfilterchangeshellcmd“.

The script:

 It will create an output file called /scripts/aliases.out every-time it runs with a bit of information in the file.

Remember, you might need to install bash, if it is not installed:

  • pkg install bash

create a directory:

  • mkdir /scripts

You can download the script “aliases.sh” here and dump it into the directory or create a script:

  • vi aliases.sh

and copy and paste the following into the script, remember to “chmod 777“, else it will not run:

#!/usr/local/bin/bash
START=`date`
echo Script starting: $START > /scripts/aliases.out

facebook=`pfctl -t facebook -T show | wc -l`
microsoft=`pfctl -t microsoft -T show | wc -l`
youtube=`pfctl -t youtube -T show | wc -l`
# Removing spaces from string:
printf -v facebook '%s' $facebook
printf -v microsoft '%s' $microsoft
printf -v youtube '%s' $youtube

if [ $facebook != 0 ]; then
 echo Facebook OK total = $facebook >> /scripts/aliases.out
else
 echo Facebook NOT OK total = $facebook >> /scripts/aliases.out
 whois -h whois.radb.net -- "-i origin AS32934" | awk "/^route:/ {print $2;}" | awk '{print $2}' | sort | uniq > /usr/local/www/aliastables/facebook.txt
 echo WhoIs completed for Facebook >> /scripts/aliases.out
 pfctl -t facebook -T replace -f /usr/local/www/aliastables/facebook.txt
 END=`date`
 echo Facebook Script ending: $END >> /scripts/aliases.out
fi

if [ $microsoft != 0 ]; then
 echo Microsoft OK total = $microsoft >> /scripts/aliases.out
else
 echo Microsoft NOT OK total = $microsoft >> /scripts/aliases.out
 whois -h whois.radb.net -- "-i origin AS8075" | awk "/^route:/ {print $2;}" | awk '{print $2}' | sort | uniq > /usr/local/www/aliastables/microsoft.txt
 echo WhoIs completed for Microsoft >> /scripts/aliases.out
 pfctl -t microsoft -T replace -f /usr/local/www/aliastables/microsoft.txt
 END=`date`
 echo Microsoft Script ending: $END >> /scripts/aliases.out
fi

if [ $youtube != 0 ]; then
 echo Youtube OK total = $youtube >> /scripts/aliases.out
else
 echo Youtube NOT OK total = $youtube >> /scripts/aliases.out
 whois -h whois.radb.net -- "-i origin AS43515" | awk "/^route:/ {print $2;}" | awk '{print $2}' | sort | uniq > /usr/local/www/aliastables/youtube.txt
 echo WhoIs completed for Youtube >> /scripts/aliases.out
 pfctl -t youtube -T replace -f /usr/local/www/aliastables/youtube.txt
 END=`date`
 echo Youtube Script ending: $END >> /scripts/aliases.out
fi

END=`date`
echo Script ending: $END >> /scripts/aliases.out

Debug errors:

  • debug your rule-set with: pfctl -f /tmp/rules.debug
  • Check  pfctl -s labels
  • Check facebook/microsoft/youtube table: pfctl -t <aliasName> -T show
  • Kill state: pfctl -k label -k “USER_RULE: <aliasName>”

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: