Sunday, September 19, 2010

Wget & apt-get behind Proxy Server

Unless you want to spy on how much traffic the proxy has seen today, it's a bad idea to hard-code its address. wpad.dat does some intelligent things like bypassing the proxy for dynamic CGIs we know can't be cached.
but it doesn't seem to me that wget understands the automatic configuration script.
So I've added the following to my ~/.wgetrc file:
http_proxy = http://donut.unet.brandeis.edu:3128/
use_proxy = on
wait = 15
apt-get behind proxy server
Acquire::http::Proxy "http://proxy.server.here:3128/";
export http_proxy=http://192.168.1.1:3128/
UPDATE – Proxy exceptions
export no_proxy="192.168.0.1,localdomain"

Squid Proxy Configuration with Authentication in lucid

This is a short guide on how to set up a transparent squid proxy server. Squid is a caching proxy for the Web supporting HTTP, HTTPS, FTP, and more. It reduces bandwidth and improves response times by caching and reusing frequently-requested web pages. Squid has extensive access controls and makes a great server accelerator

Installing Squid

apt-get install squid3

Edit the squid config file.

sudo nano /etc/squid3/squid.conf

acl localnet src 192.168.1.0/255.255.255.255

#Default:
 http_access allow all

# Squid normally listens to port 3128
http_port 3128

# change default cache size
cache_dir ufs /var/spool/squid3 1024 16 256

# enables access logging
access_log /var/log/squid3/access.log squid

# enables cache logging
 cache_log /var/log/squid3/cache.log
# enables cache store logging
 cache_store_log /var/log/squid3/store.log
# change the admin contact details
 cache_mgr admin@abc.com
 cache_effective_user proxy
#Default hostname change
 visible_hostname squidsvr

You can configure squid to prompt users for a username and password. Squid comes with a program called ncsa_auth that reads any NCSA-compliant encrypted password file.If you wish to use authentication with your proxy you will need to install apache2 utilities
sudo aptitude install apache2 apache2-utils
touch /etc/squid3/squid.passwd
chmod o+r /etc/squid3/squid.passwd
To add your first user you will need to specify -c

sudo htpasswd -c /etc/squid3/squid.passwd first_user
Thereafter you add new users with
sudo htpasswd /etc/squid3/squid.passwd another_user
Edit the squid config file

sudo vi /etc/squid3/squid.conf
 Locate nsca_auth authentication helper

Usually nsca_auth is located at /usr/lib/squid/ncsa_auth. You can find out location using rpm (Redhat,CentOS,Fedora) or dpkg (Debian and Ubuntu) command:
# dpkg -L squid3 | grep ncsa_auth
Output:
/usr/lib/squid3/ncsa_auth

If you are using RHEL/CentOS/Fedora Core or RPM based distro try:
# rpm -ql squid3 | grep ncsa_auth
Output:
/usr/lib/squid3/ncsa_auth

Configure nsca_auth for squid proxy authentication

Edit squid.conf; specifically, you need to define the authentication program in squid.conf, which is in this case ncsa_auth. Next, create an ACL named ncsa_users with the REQUIRED  that forces Squid to use the NCSA auth_param method you defined previously. Finally, create an http_access entry that allows traffic that matches the ncsa_users ACL entry. Here’s a simple user authentication example; the order of the statements is important:

# vi /etc/squid/squid.conf
Append (or modify) following configration directive:

  auth_param basic program /usr/lib/squid3/ncsa_auth /etc/squid3/squid.passwd

auth_param basic children 5
auth_param basic realm Squid proxy-caching web server
auth_param basic credentialsttl 2 hours
auth_param basic casesensitive off

acl users proxy_auth REQUIRED
acl manager proto cache_object
acl sectionx proxy_auth REQUIRED
acl purge method PURGE

#Default:
 http_access allow users

 auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid3/squid.passwd : Specify squid password file and helper program location
  • auth_param basic children 5 : The number of authenticator processes to spawn.
  • auth_param basic realm Squid proxy-caching web server : Part of the text the user will see when prompted their username and password
  • auth_param basic credentialsttl 2 hours : Specifies how long squid assumes an externally validated username:password pair is valid for - in other words how often the helper program is called for that user with password prompt. It is set to 2 hours.
  • auth_param basic casesensitive off : Specifies if usernames are case sensitive. It can be on or off only
  • acl ncsa_users proxy_auth REQUIRED : The REQURIED term means that any authenticated user will match the ACL named ncsa_users
  • http_access allow ncsa_users : Allow proxy access only if user is successfully authenticated.
It should look like

#Recommended minimum configuration per scheme:
auth_param basic children 5
auth_param basic realm Squid proxy-caching web server
auth_param basic credentialsttl 2 hours
auth_param basic casesensitive off

#Recommended minimum configuration:
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32
#
# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
#acl localnet src 10.0.0.0/8    # RFC1918 possible internal network
#acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network

acl users proxy_auth REQUIRED
acl manager proto cache_object
acl users proxy_auth REQUIRED
acl sectionx proxy_auth REQUIRED
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT
Restart squid:
# /etc/init.d/squid3 restart

Now user is prompted for username and password.
Proxy server log Analyzer (Webalizer)

apt-get install webalizer
To configure the software you must look at the global file /etc/webalizer.conf.
Configure the /etc/webalizer.conf file.
LogType cif change to
LogType squid
LogFile /var/log/apache2/access.log
The option LogFile specifies the logfile to use with Webalizer. The default log file is supposed to be the access_log of Apache Web Server, but you can specifies a different one, like the one Squid Proxy Server makes named access.log if you use it in httpd-accelerator mode.

LogFile /var/log/apache2/access.log.1 change to
LogFile /var/log/squid3/access.log

OutputDir /home/www/webalizer
The option OutputDir specifies the location of the output directory to use for the reports of Webalizer. All present and future report files generated by the Webalizer program will be hosted in this directory. It is recommended that you create this directory where your Apache web site resides.
Incremental yes
The option Incremental if set to Yes tells the program only to process partial logs file, and allows you to rotate your log files as much as you want without the loss of access information. It's recommended to set this option to Yes.
Un comment the following entry
#HistoryName     webalizer.hist
IncrementalName webalizer.current
HostName PXESVR
DNSCache        dns_cache.db
DNSChildren     20
Quiet           no
TimeMe          yes
VisitTimeout    1800
CountryGraph    yes
DailyGraph      yes
DailyStats      yes
GraphLegend     yes
GraphLines      2
TopSites        30
TopKSites       10
TopURLs         30
TopKURLs        10
TopReferrers    30
TopAgents       15
TopCountries    30
TopEntry        10
TopExit         10
TopSearch       20
TopUsers        20
After configuring all the required options .Now you need to restart the Apache2 server using

/etc/init.d/apache2 restart

Running Webalizer manually first time
Now it's time to run the program to generate reports, html and graphics in the default Webalizer directory so that we can see them in our web browser interface. This step is required just the first time you install and use Webalizer, since it's preferable to use a cron job to automate this task in the future. To run Webalizer manually, to generate reports, use the following command:

        [root@test ] /# /usr/bin/webalizer
        Webalizer V2.21-02 (Linux 2.6.32) English
        Using logfile /var/log/apache2/access.log
        Creating output in /home/www/webalizer
        Hostname for reports is 'sites.test.com'
        History file not found...
        Previous run data not found...
        Saving current run data... [20/09/2010 04:30:03]
        Generating report for September 2010
        Generating summary report
        Saving history information...
        81 records (2 ignored) in 0.31 seconds

At this stage, we should verify that Webalizer is working on the system. To do that, point your web browser to the following address: http://my-web-server/webalizer/. The my-web-server is the address where your Apache web server lives, and usage is the directory that host all the Webalizer reports files.

Saturday, September 18, 2010

Disable GDM at bootup on Ubuntu Lucid

on ubuntu 10.04

Ubuntu lucid replaces the old init system with upstart, this means you can't use update-rc.d gdm -f remove  to stop it starting up at bootup. So to disable it we need to edit the upstart files in /etc/init (note not /etc/init.d!) thanks to sisco311 on the ubuntu forums the solution is to edit /etc/init/gdm.conf and change

start on (filesystem
          and started hal
          and tty-device-added KERNEL=tty7
          and (graphics-device-added or stopped udevtrigger))
stop on runlevel [016]

to

start on (runlevel [3]
          and filesystem
          and started hal
          and tty-device-added KERNEL=tty7
          and (graphics-device-added or stopped udevtrigger))
stop on runlevel [0126]

Then reboot!

You can start gdm via service gdm start

on Ubuntu 10.04.1

booting with the text kernel parameter will also prevent display managers managed by Upstart (e.g. gdm, kdm and lxdm) from being started at boot time.
If you are using Grub2, then in /etc/default/grub replace:
Code:
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
with
Code:
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash text"
then run:
Code:
sudo update-grub
Remove splash to disable the splash screen and/or quiet to make the boot process verbose.