Bash scripts to find out missing lines and common lines between two files

Sometime back I had explained how to find the difference and common lines in two files, Using that giving you 2 simple (very simple) scripts

1. Find out the missing lines between two files

cat diff.sh

#!/bin/bash
if [ $# -eq 0 ];then
        echo "No arguemnts given; Usage : ./diff file2 file1"
        exit 1
fi
echo "Lines exist in $2 and not in $1 are :"
grep -Fxv -f $1 $2

2. Find out the common lines between two files

cat common.sh

#!/bin/bash
if [ $# -eq 0 ];then
        echo "No arguments given; Usage : ./common file1 file2"

        exit 1
fi
echo "Common files in $1 and $2 are :"
awk 'NR==FNR{a[$1]++;next} a[$1] ' $1 $2

Enjoy 🙂

Advertisements

Python script to check ssh login on multiple hosts against a common password

Below python script can be used to check ssh login on multiple hosts against a common password and print the output to different files based on status. It prompts user to enter the password and check it against all the hostname in the input_file. To speed up the process, I’m using multithreading module which run the jobs in parallel to accomplish the task faster. This script works with Python3.5 version

#!/bin/python3
import threading, time, paramiko, socket, getpass
from queue import Queue
locke1 = threading.Lock()
q = Queue()

#Check the login
def check_hostname(host_name, pw_r):
    with locke1:
        print ("Checking hostname :"+str(host_name)+" with " + threading.current_thread().name)
        file_output = open('output_file','a')
        file_success = open('success_file','a')
        file_failed = open('failed_file','a')
        file_error = open('error_file','a')
        ssh = paramiko.SSHClient()
        ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
        try:
          ssh.connect(host_name, username='root', password=pw_r, timeout=5)
          #print ("Success")
          file_success.write(str(host_name+"\n"))
          file_success.close()
          file_output.write("success: "+str(host_name+"\n"))
          file_output.close()

          # printing output if required from remote machine
          #stdin,stdout,stderr = ssh.exec_command("hostname&&uptime")
          #for line in stdout.readlines():
           # print (line.strip())

        except paramiko.SSHException:
                # print ("error")
                file_failed.write(str(host_name+"\n"))
                file_failed.close()
                file_output.write("failed: "+str(host_name+"\n"))
                file_output.close()
                #quit()
        except paramiko.ssh_exception.NoValidConnectionsError:
                #print ("might be windows------------")
                file_output.write("failed: " + str(host_name + "\n"))
                file_output.close()
                file_failed.write(str(host_name+"\n"))
                file_failed.close()
                #quit()
        except socket.gaierror:
          #print ("wrong hostname/dns************")
          file_output.write("error: "+str(host_name+"\n"))
          file_output.close()
          file_error.write(str(host_name + "\n"))
          file_error.close()

        except socket.timeout:
           #print ("No Ping %%%%%%%%%%%%")
           file_output.write("error: "+str(host_name+"\n"))
           file_output.close()
           file_error.write(str(host_name + "\n"))
           file_error.close()

        ssh.close()


def performer1():
    while True:
        hostname_value = q.get()
        check_hostname(hostname_value,pw_sent)
        q.task_done()

if __name__ == '__main__':

    print ("This script checks all the hostnames in the input_file with your standard password and write the outputs in below files: \n1.file_output\n2.file_success \n3.file_failed \n4.file_error \n")

    f = open('output_file', 'w')
    f.write("-------Output of all hosts-------\n")
    f.close()
    f = open('success_file', 'w')
    f.write("-------Success hosts-------\n")
    f.close()
    f = open('failed_file', 'w')
    f.write("-------Failed hosts-------\n")
    f.close()
    f = open('error_file', 'w')
    f.write("-------Hosts with error-------\n")
    f.close()

    with open("input_file") as f:
        hostname1 = f.read().splitlines()

#Read the standard password from the user
    pw_sent=getpass.getpass("Enter the Password:")
    start_time1 = time.time()

    for i in hostname1:
        q.put(i)
    #print ("all the hostname : "+str(list(q.queue)))
    for no_of_threads in range(10):
        t = threading.Thread(target=performer1)
        t.daemon=True
        t.start()

    q.join()
    print ("Check output files for results")
    print ("completed task in" + str(time.time()-start_time1) + "seconds")

Run a script at boot time in CentOS 7

By default /etc/rc.local and /etc/rc.d/rc.local are no longer executable in CentOS7 with the new systemd-changes. Follow the below steps to make the script /root/g.sh run at boot time:

1. chmod +x /etc/rc.d/rc.local
2. chmod +x /root/g.sh
2. Mention your script at the bottom of the file /etc/rc.local (/etc/rc.local is a symlink to /etc/rc.d/rc.local)as below :

sh /root/g.sh

Restart and check 🙂

Get notification after every successful backup from BackupPC

Edit the configuration in BackupPC is as below :

Edit config –> Backup Settings –> User commands -> DumpPostUserCmd as
/theg/email.sh $xferOK $host $type $client $hostIP $share $XferMethod $sshPath $cmdType
where /theg/email.sh is the custom script.

Below are the contents of the email.sh script :

 

#!/bin/bash
# script to send simple email
# Email To ?
EMAIL=”your_email_address”
# Email text/message
EMAILMESSAGE=”/tmp/emailmessage.txt”

# Grab the status variables
xferOK=$1
host=$2
type=$3
client=$4
hostIP=$5
share=$6
XferMethod=$7
sshPath=$8
cmdType=$9

# Check if backup succeeded or not.
if [[ $xferOK == 1 ]]; then
        STATUS=”SUCCESS”
else
        STATUS=”FAILED”
fi

# email subject
SUBJECT=”[BackupPC] $STATUS for host: $client”

# Email text/message
echo “Filesystem backup for $host $STATUS” > $EMAILMESSAGE
echo “” >>$EMAILMESSAGE
echo “Type: $type” >>$EMAILMESSAGE
echo “Client: $client” >>$EMAILMESSAGE
echo “Host: $host” >>$EMAILMESSAGE
echo “Host IP: $hostIP” >>$EMAILMESSAGE
echo “Share: $share” >>$EMAILMESSAGE
echo “XferMethod: $XferMethod” >>$EMAILMESSAGE
echo “sshPath: $sshPath” >>$EMAILMESSAGE
echo “cmdType: $cmdType” >>$EMAILMESSAGE

# send an email using /bin/mail
/bin/mail -s “$SUBJECT” “$EMAIL” < $EMAILMESSAGE

Enjoy 🙂