Author: K.M. Ali Qamar

How to create SFTP shared folder so multiple restricted and chroot Jailed users can access the same folder

You need to migrate windows based SFTP server to the RHEL platform to save cost and make it easy to manage.

To build a similar kind of environment as I have on windows, I need to create three types of user accounts, and as you know, chroot jailed means that the account is jailed and can’t get out of its home directory by ensuring no other users are affected. So this makes it a bit challenging.

Based on my requirement,

  1. Users should be restricted to their directories and not see OS directories, ensuring no other users are affected. 
  2. Some users will have FULL access to other user’s home directories.
  3. Some users will have read-only access to some other user’s fully shared directories.

There will be other solutions to fix this problem, but I am doing the easiest way I can think of.

I am going to discuss three scenarios.

Scenario #1: Create three SFTP jailed Chroot accounts, but one account should access the files of the other two accounts’ home directory.

I have created a folder structure as shown in the below table. You can see user03 home directory is one level up from the other two accounts.

UsersHome directories
User01/data/accounts/user01
User02/data/accounts/user02
accounts/data/accounts

create directories

mkdir -p /data/accounts/user01 ; mkdir -p /data/accounts/user02

Make the landing directories home directories.

create logins and directories, and you can also amend the /etc/passwd if you have created the user accounts and not added the home directories.

useradd  -d /data/accounts/user01 -s /sbin/nologin user01 ; useradd  -d /data/accounts/user02 -s /sbin/nologin user02

Make sure you have settled the password of your accounts

passwd <userName>

Add your account into the group named sftpusers

groupadd sftpusers 
usermod -aG sftpusers user01

Setup the appropriate permissions

chown user01:sftpusers /data/accounts/user01/internal ; chmod -R 755 /data/user01/internal
chown user02:sftpusers /data/accounts/user01/internal ; chmod -R 755 /data/user02/internal

Edit the sshd_config file

Edit the /etc/ssh/sshd_config and add the following lines.

Configure /etc/ssh/sshd_config

#Subsystem      sftp    /usr/libexec/openssh/sftp-server
Subsystem       sftp    internal-sftp

# BEGIN SFTP-Server sftpusers block

Match Group sftpusers
ChrootDirectory  %h
AllowTcpForwarding no
ForceCommand internal-sftp
X11Forwarding no

#End group sftpusers configuration

Restart the SSHD service

systemctl restart sshd

You have configured user01 and user02, and you can login and upload the files.
Let’s start with our third account named “accounts,” This account should have FULL access to the user01/user02 files.

Configure your third user accounts

useradd  -d /data/accounts -s /sbin/nologin accounts ; mkdir -p  /data/account/

make accounts member of group sftpusers

usermod -aG sftpusers accounts

Make sure you give good permissions to the home directories of user01/user02

chown user02:sftpusers /data/accounts/user01/internal ; chmod -R 775 /data/user02/internal

change the /etc/ssh/sshd_config to add the user accounts

# BEGIN SFTP-Server "accounts" block

Match user accounts
ChrootDirectory  /data/accounts
AllowTcpForwarding no
ForceCommand internal-sftp
X11Forwarding no
#END SFTP-Server accounts block

Restart the SSHD service.

systemctl restart sshd

TESTING & Diagnostic Steps

you can test your login by doing: sftp user05@localhost
Check /var/log/secure for any errors with permissions and sftp.

sftp and/or scp may fail at connection time if you have shell initialization (.profile, .bashrc, .cshrc, etc) which produces output for non-interactive sessions. This output confuses the sftp/scp client. You can verify if your shell is doing this by executing this

ssh <yourhost> /usr/bin/true

Scenario # 2 one folder is shared by multiple chroot jailed accounts

UsersHome directories
User03/dpt/files
User04/dpt/files

As shown above, both chroot jailed users have shared folders, so we will create the users and configure them.

mkdir -p /dpt/files
useradd  -d /dpt/files -s /sbin/nologin user03
useradd  -d /dpt/files -s /sbin/nologin user04

create group grp-shared and add your accounts into the newly created group.

groupadd grp-shared
usermod -aG grp-shared user03 ; usermod -aG grp-shared user04

To check that your users have the desired group.

groups user03

Make sure you give good permissions to the home directories of user01/user02

chgrp  grp-shared /dpt/files/internal ; chmod -R 775 /dpt/files/internal

change the /etc/ssh/sshd_config to add your group grp-shared in the sshd configuration

#BEGIN SFTP-Server grp-shared block

Match Group grp-shared
ChrootDirectory  /dpt/files
AllowTcpForwarding no
ForceCommand internal-sftp
X11Forwarding no
PermitTunnel no
PasswordAuthentication yes
#END SFTP-Server grp-shared block

Restart the SSHD service.

systemctl restart sshd

Scenario # 3 single folder is shared by multiple chroot jailed accounts, but one user has read-only access to that shared folder

UsersHome directories
user03/dpt/files
user04/dpt/files
user05/dpt/files ( READ-ONLY ACCESS)

Here I will show you the configurations for the third user, which has only READ-ONLY access because shared folder scenario I have discussed above. 

useradd   -s /sbin/nologin user05
passwd user05

You can have the default home directory to the users as /home/user05, but in the sshd_config file, you can chroot directory to the /dpt/files.

Match User user05
ChrootDirectory /dpt/files
AllowTcpForwarding no
ForceCommand internal-sftp
X11Forwarding no
PermitTunnel no
PasswordAuthentication yes

Restart the SSHD service.

systemctl restart sshd

Disclaimer: All information posted is merely for educational and informational purposes. Should you decide to act upon any information on this article, you do so at your own risk.

How to find the biggest files in filesystem – Linux, UNIX, HP-UX

Disk space issues are the most common problems raise in the day-to-day life of Linux system admin. So in this article, you can find out the commands to find out the largest files in your file system which are causing problems for the filesystem.

Find for Large Files under Specific Mountpoint

find /var -xdev -type f -size +500000c -exec ll {} ; | sort -nk 5
find /home -xdev -type f -size +5000000c -exec ll {} ; | sort -nk 5
find /opt -xdev -type f -size +5000000c -exec ll {} ; | sort -nk 5
find /usr -xdev -type f -size +5000000c -exec ll {} ; | sort -nk 5
find /tmp -xdev -type f -size +50000000c -exec ll {} ; | sort -nk 5
find /var -xdev -type f -size +50000000c -exec ll {} ; | sort -nk 5

For Solaris & Linux, replace ‘ll’ with ‘ls -l’

find / -xdev -type f -size +50000000c -exec ls -l {} ; | sort -nk 5
find /usr -xdev -type f -size +50000000c -exec ls -ld {} ; | sort -nk 5
find /var -xdev -type f -size +50000000c -exec ls -ld {} ; | sort -nk 5
find / -xdev -type f -size +50000000c -exec ls -ld {} ; | sort -nk 5
find /nfs/tmp -xdev -type f -size +5000000c -exec ls -ld {} ; | sort -nk 5
find /data -xdev -type f -size +50000000c -exec ls -ld {} ; | sort -nk 5
find /usr1/data1 -xdev -type f -size +50000000c -exec ls -ld {} ; | sort -nk 5
find /apps -xdev -type f -size +50000000c -exec ls -ld {} ; | sort -nk 5
find /nfsdata -xdev -type f -size +50000000c -exec ls -l {} ; | sort -nk 5
find /see_data -xdev -type f -size +50000000c -exec ls -l {} ; | sort -nk 5

HP-UX

bdf /var

Solaris, Linux, and AIX

df -k /var

Where -k report size in KB

df -h /var

-h report size in GB / Human-readable

df -h /var

size in GB (Solaris 9++)

Linux interview questions

Linux Interview Questions From Real Interviewer

Before we start a technical side, I think the first step to winning an interview is to go through with few things that will help you be successful at the interview.

Analyzing The Position:

It would be best if you began with a systematic review of the job description. Your objective is to collect as much information as possible about how the job and what knowledge and competencies are needed to perform your future job.

Take a close look at the official job description. Consider the environment in which the job is going to be performed. Are there any special skills required? what tools and software you need to know to perform this job.

Make a note of any tools that are being used to perform the tasks of the job, For instance, with the kind of servers, storage protocols, and operating systems they are using, and familiarise yourself with all those tools before the interview.

You can also talk with those who have held the job in the past. If you don’t know anyone, just try to find them through linkedin.com and scan their profiles for tools and skillsets they have used.

Check what competencies and skills contributed to their success? Did the lack of specific competencies or skills cause or contribute to difficulties that they experienced on the job?

What you should have and what interviewers are looking for:

Before they offer you the contract for a long-term relationship with you, they want to know what’s in it for them.

  • Problem-solving skills. It would be best if you have a couple of stories in which you solved the problem independently. Bring any certificate if you got any. 
  • Good people skills. The ability to foster good working relationships is a vital business skill. Discuss situations where you worked well with others. For instance, you can narrate your stories of helping co-workers under challenging conditions.
  • Getting Things Done Skills. The ability to get things done. Mention those times when your good judgment or ingenuity helped you close a deal.
So let’s start our main topic for which you are visiting this post.

1. Tell us about your experience with Linux and the projects you have accomplished?

In response to this question, you should tell your experiences, such as your current or previous job experiences and all the big/small projects you have done. Don’t trust your present memory. Please write it down and try to be fluent in these details.

2. What is the difference between Linux & Unix?

Unix: It is considered the mother of most operating systems. It is a Proprietary operating system and original code developed by AT & T. Originally the Bourne shell. Compatible with PA and Itanium machines. It is also compatible with many command interpreters. AIX and BSD are the renowned flavors of UNIX.
    Linux: The code developed by Linus Torvalds. By default, the shell is BASH (Bourne Again Shell) and can use multiple shells. Ubuntu, Debian , centos, and Redhat are the different flavors of Linux.

      3. What is the shell?

      I think the simplest answer is  “shell is a language to communicate with a computer.”Bash or the Bourne-Again Shell is the most widely used shell among Linux admins.  

      4. What is the structure of Linux filesystem?

      Linux interview Questions form Real interviewer

      All files on a Linux system are stored on file systems, which are inverted into a single inverted tree of directories, known as a file-system hierarchy. This tree is invented because the root of the tree is to be at the top of the hierarchy, and the branches of directories and subdirectories stretch below the root. 
      LOCATIONPURPOSE
      /Is the root directory at the top of the file-system hierarchy.
      /usrIt contains installed software, shared libraries, includes files and read-only program data. Subdirectories include /usr/bin user commands, /usr/sbin/ system admin commands, /usr/local: locally customized software
      /etcconfiguration files specific to this system
      /varvariable data specific to this system that should persist between boots. Files that dynamically change such as databases, cache directories, log files, and website content the most famous one
      /runRuntime data for processes started since the last system boots.
      /homeHome directories are where regular users store their personal data and configuration files
      /tmpThis directory has a world-writable space. Any user can put files here, and within 30 days, the files will be deleted.

      5. Which directory contains regular commands and utilities?

      /usr contains regular commands and utilities.

      6. What is the root account?

      The root account is a system administrator account and has all the powers to give you complete control of the system. Here you can create and maintain user accounts, assigning different permissions for each account. It is the default account every time you install Linux.

      5. How to ensure that newly created users have passwords that must be changed every 90 days?

      you can set password policy from the file /etc/login.defs by changing the value ” PASS_MAX_Days” to 90.

      6. How you will configure the administrative rights for all members of the group named consultants that its members are able to execute any command as any user?

      Create the new file /etc/sudoers.d/consultants and add the following content to it. You can use the sudo vim /etc/sudoers.d/consultants command to perform this step.%consultants ALL=(ALL) ALL

      7. What are symbolic links?

      Symbolic links are files that act as pointers to other files.

      8. If you lost your root password, can you change the root password, and how?

      We can reset the root password, and you can look into this link for the details.

      9. What is initialzation?

      The term “init” is the program on Unix-like system which spawns all other processes. It runs as a daemon and typically has a PID 1.

      10. What is a pstree?

      This is the command to check the running processes.

      11.What will you do if you receive the alert that your partition is full?

      First of all, check which files are the biggest ones on the affected partition. In most cases, log files are there which are causing the troubles, so in this case, you can zip the file after nullifying.

      12. How can you find out that how much memory is being used by your server?

      You can take the help of different tools to find the memory usage on your system.
      free - m
      
      vmstat
      
      top
      
      htop

      17. What is a recommendation of the SWAP partition under a Linux system?

      The recommended size for a swap partition is twice the amount of physical memory available on the system. However, if you cant put the double in size, you can put the same amount as memory in your system.

      18. How to display IP address?

      ip address show

      19. How to displaying the Network performance Statistics?

      ip -s link show ens3       #ens3 is your interface

      20. How to displaying the Routing table?

      ip route

      21. How to Tracing Route Taken by Traffic?

      tracepath access.redhat.com tracepath options ICMP ( -I) TCP (-T)*traceroute command not installed by default

      22. Troubleshooting Ports and Services?

      ss command is meant to replace netstat
      ss -ta
      -n show numbers insted of names for interfaces and ports
      -t show TCP sockets
      -u Show UDP sockets
      -l show listening sockets
      -a show all ( listening and establisted)

      23. How to show the status of the network devices?

             nmcli dev status

      24. How to show the connection status?

               nmcli con show

      25. How to list on active connection?

           nmcli con show --active

      26. How to control network connections?

      You must be able to demonstrate how to control your network connections.
      nmcli con up static-ens3
      nmcli dev dis ens3
      nmcli con show static-ens3
      nmcli con reload
      nmcli con down "static-ens3"
      nmcli con up "static-ens3"

      27. How to remove a network connection?

      nmcli con del static-ens3

      28. What are nmcli commands?

      nmcli dev status : show the networkManagaer stusus of all netwokr interfaces
      nmcli con show : List all connections
      nmcli con show NAME : list the current settings for the connection name
      nmcli con add con-name name Add a new connection named name
      nmcli con mod name modify the connectiuon anem
      nmcli con up name enable it and make it up
      nmcli dev dis devf disconnects it.
      nmacli con del name Deletes

      29. How to change the System hostname?

      hostnameclt set-hostname host@example.com
      hostnamectl status
      hostname

      30. How to test the DNS name resolution?

      host classroom.example.com
      host IP

      31. How to configure the name resolution without the DNS server?

      Command to check the hostname in Redhat is getent hosts hostnameIf you don’t have a DNS server you can configure it through /etc/hosts.getent hosts hostname command can be used to test the hostname resolution using the /etc/hosts file.

      32. How to Manage SELinux?

      Selinux is a set of security rules that determine which process can access which files, directory, and ports. every file process directory and port has a special security label called an SELinux context.By default, policy does not allow any interaction.SELinux has several contexts: user, role, type, and sensitivity.

      33. How to find out the current SELinux policy?

      [root@servera ~]# grep '^SELINUX' /etc/selinux/config
      SELINUX=enforcing
      SELINUXTYPE=targeted
      *if you want to disable you, then replace enforcing to disabled and reboot the server.

      34. How to Change SELinux context?

      This is how you can change the /virtual context
      semanage fcontext -a -t httpd_sys_content_t '/virtual(/.*)?'
      restorecon -RFvv /virtual

      35. How to check the current context?

      ls -zD /virtual OR ls -Zd /var/www
       

      36. How are you troubleshooting SELinux issues?

      You can check the logs from autdit.log
      tail /var/log/audit/audit.log
      
      And do,
      
      less /var/log/messages and search like this / sealert and copy past sealert -l b1c9cc8f-a953-4625-b79b-82c4f4f1fee3
      
      ausearch -m AVC -ts recent
      
      -ts time based search

      39. How to check if ports are open or not?

      There are many ways to perform this activity, but few famous are shown below:by using /etc/services
      # cat /etc/services | grep " 123/"
      Diagnostic Steps
      by using netstat
      # netstat -tulpn | grep :123
      udp 0 0 0.0.0.0:123 0.0.0.0:* 26868/chronyd
      udp6 0 0 :::123 :::* 26868/chronyd
      By using nmap  (nmap is a utility for network discovery and security auditing)
      # nmap -p 123 <server-ip>
      By default nmap would give details of TCP port 123. To get details of 123/udp port use below command :
      Raw
      # nmap -sU -p 123 <server-ip>

      41. what is systemd?

      systemd uses units to manage different types of objects.

      41. What is service units?

      service units have service extensions and represent system services, for example web servers.

      42. What are socket units?

      socket units have a .socket extension and represent inter-process communication sockets that systemd should monitor.

      43. What is path units?

      Path units have .path extension and are used to delay the activation of service until a specific file system change occurred.

      44. What is systemctl?

      It is used to manage units. you can get more information with systemctl -t help

      45. How to check all the services and their status?

      systemctl list-units --type=service

      46. How to view the status of the service in Linux?

      systemctl status name.type
      
      systemctl status sshd.service
      
      systemctl status chronyd

      47. How to verify the status of the service?

      systemctl is-active sshd.service

      47. How to check if service is enabled during boot?

      systemctl is-enabled sshd.service

      48. How to check if the service is failed during boot?

      systemctl is-failed sshd.service

      49. how to display to check all the services failed during boot?

      systemctl is-failed –type=service 

      50. How you define the booting process?

      • The server is powered on. The system firmware runs a POST  (UFFI or BIOS) and stats to initialize the hardware.
      • System firmware searches for a bootable device.
      • System firmware reads a boot loader from the disk and then passes control of the system to the boot loader.
      • In REDHAT boot loader is GRand Unified bootloader version 2 (GRUB2) which loads configuration from this file ( /boot/groub2/grub.cfg).
      • Now, you will have the option to select the kernel and once you select the kernel, the boot loader loads the kernel and Initramfs from the disk.
      • boot loader hands over the kernel to check the location of the intramfs in the memory. Here kernel initiates all the hardware by finding out the drivers.
      • systemd instance from intiramfs executes all units for the initrd.target.

      51. What are the system Targets modes

      systemd targets are represented by targets units, but in earlier versions, prior to RHEL 7, the system was coming with SysV runlevels.

      52. how to check the default target?

      systemctl get-default

      53. how to view the target units?

      systemctl list-units --type target --all

      54. How to change the default target?

      systemctl get-default
      
      systemctl set-default multi-user.target
      
      reboot

      55. How to boot in the rescue mode?

      systemctl rescue

      56. How to check the available subscription?

      subscription-manager list --available | less

      57. How to attach the subscription?

      subscription-manager attach --auto
      The most important yum command
      yum help
      yum list 'http*'
      yum search all 'web server'
      yum info httpd
      yum provides /var/www/html
      yum list kernel
      yum remove httpd
      
      yum group list
      yum group "RPM Development Tools"
      
      
      

      58. All the rpm-related history save in which file?

      /var/log/dnf.rpm.log or yum history

      59. At what location system save your repo files?

      /etc/yum.repos.d/errata.repo

      60. How to check which repos are attached to the system? 

      yum repolist all
                

      How To Monitor VMware envirnment with Grafana

      This step-by-step guide uses the Official telegraph vSphere plugin to pull metrics from vCenter. We will pull metrics such as compute, network and storage resources. Before starting with this guide, I assume you have a freshly installed operating system, ubuntu 20. so let’s being with our work.

      Step: 1 Install Grafana on Ubuntu

      This tutorial tested on freshly installed OS Ubuntu 20.04.
      • Start your Grafana installation.
      wget https://dl.grafana.com/oss/release/grafana_7.1.3_amd64.debsudo dpkg -i grafana_7.1.3_amd64.deb
      • Now start and enable your Grafana service.
      sudo systemctl start grafana-server.servicesudo systemctl enable grafana-server.service
      • Check Grafana service status.
      sudo systemctl status grafana-server.service
      • At this point, Grafana is installed, and you can log in to your Grafana by following
      url: http://[your Grafana server ip]:3000The default username/password is admin/admin
      • Upon the first login, Grafana will ask you to change the password.
      • Be careful HTTP is not a secure protocol. You can further secure it by putting SSL certificates.

      Step: 2 Install Influx DB

      • Inquire about the available InfluxDB version in your apt-cache by the following command.
      sudo apt-cache policy influxdbIt will be the last stable version of InfluxDB. We will use a later version 1.8 of InfluxDB, so we will update the apt cache first and add the required information to the repository.wget -qO- https://repos.influxdata.com/influxdb.key | sudo apt-key add -source /etc/lsb-releaseecho "deb https://repos.influxdata.com/${DISTRIB_ID,,} ${DISTRIB_CODENAME} stable" | sudo tee /etc/apt/sources.list.d/influxdb.listsudo apt updatesudo apt-cache policy influxdbsudo apt updatesudo apt-cache policy influxdbsudo apt install influxdb -y
      • Check the status and ensure that it sustains over the reboot.
      sudo systemctl start influxdbsudo systemctl status influxdbsudo systemctl enable influxdb
      • The InfluxDB will listen on port 8086, and if your server is on the internet, then depending on any existing firewall rules, anybody may be able to query the server using the URL
      https://[your domain name or ip]:8086/metrics
      • On my local machine where I am doing this test, is not having any firewall enabled, but if you have allowed or using public IPs, you can prevent direct access by doing these commands
      iptables -A INPUT -p tcp -s localhost --dport 8086 -j ACCEPTiptables -A INPUT -p tcp --dport 8086 -j DROP

      Step: 3 Install Telegraf

      • Now we are going to install telegraf.
      sudo apt install telegraf -y
      • Start Telegraf and ensure it starts in case of reboot.
      sudo systemctl start telegrafsudo systemctl status telegrafsudo systemctl enable telegraf
      • Configure Telegraf to pull Monitoring metrics from vCenter, so here we will configure Telegraf main configuration file:
      • In this /etc/telegraf/telegraf first, you need to add information for influxdb.
      • change your influxdb credentials.
      ————————————————————————————————————————————–[[outputs.influxdb]] urls = ["http://<Address_of_influxdb_server>:8086"] database = "vmware" timeout = "0s"#only with if you are using authentication for DB#username = "USERNAME_OF_DB"#password = "PASSWD_OF_DB"————————————————————————————————————————————-# Read metrics from VMware vCenter [[inputs.vsphere]] ## List of vCenter URLs to be monitored. These three lines must be uncommented ## and edited for the plugin to work. vcenters = [ "https://<vCenter_IP>/sdk" ] username = "administrator@vsphere.local" password = "PASSWD" # ## VMs ## Typical VM metrics (if omitted or empty, all metrics are collected) vm_metric_include = [ "cpu.demand.average", "cpu.idle.summation", "cpu.latency.average", "cpu.readiness.average", "cpu.ready.summation", "cpu.run.summation", "cpu.usagemhz.average", "cpu.used.summation", "cpu.wait.summation", "mem.active.average", "mem.granted.average", "mem.latency.average", "mem.swapin.average", "mem.swapinRate.average", "mem.swapout.average", "mem.swapoutRate.average", "mem.usage.average", "mem.vmmemctl.average", "net.bytesRx.average", "net.bytesTx.average", "net.droppedRx.summation", "net.droppedTx.summation", "net.usage.average", "power.power.average", "virtualDisk.numberReadAveraged.average", "virtualDisk.numberWriteAveraged.average", "virtualDisk.read.average", "virtualDisk.readOIO.latest", "virtualDisk.throughput.usage.average", "virtualDisk.totalReadLatency.average", "virtualDisk.totalWriteLatency.average", "virtualDisk.write.average", "virtualDisk.writeOIO.latest", "sys.uptime.latest", ] # vm_metric_exclude = [] ## Nothing is excluded by default # vm_instances = true ## true by default # ## Hosts ## Typical host metrics (if omitted or empty, all metrics are collected) host_metric_include = [ "cpu.coreUtilization.average", "cpu.costop.summation", "cpu.demand.average", "cpu.idle.summation", "cpu.latency.average", "cpu.readiness.average", "cpu.ready.summation", "cpu.swapwait.summation", "cpu.usage.average", "cpu.usagemhz.average", "cpu.used.summation", "cpu.utilization.average", "cpu.wait.summation", "disk.deviceReadLatency.average", "disk.deviceWriteLatency.average", "disk.kernelReadLatency.average", "disk.kernelWriteLatency.average", "disk.numberReadAveraged.average", "disk.numberWriteAveraged.average", "disk.read.average", "disk.totalReadLatency.average", "disk.totalWriteLatency.average", "disk.write.average", "mem.active.average", "mem.latency.average", "mem.state.latest", "mem.swapin.average", "mem.swapinRate.average", "mem.swapout.average", "mem.swapoutRate.average", "mem.totalCapacity.average", "mem.usage.average", "mem.vmmemctl.average", "net.bytesRx.average", "net.bytesTx.average", "net.droppedRx.summation", "net.droppedTx.summation", "net.errorsRx.summation", "net.errorsTx.summation", "net.usage.average", "power.power.average", "storageAdapter.numberReadAveraged.average", "storageAdapter.numberWriteAveraged.average", "storageAdapter.read.average", "storageAdapter.write.average", "sys.uptime.latest", ] # host_metric_exclude = [] ## Nothing excluded by default # host_instances = true ## true by default # ## Clusters cluster_metric_include = [] ## if omitted or empty, all metrics are collected # cluster_metric_exclude = [] ## Nothing excluded by default # cluster_instances = false ## false by default # ## Datastores datastore_metric_include = [] ## if omitted or empty, all metrics are collected # datastore_metric_exclude = [] ## Nothing excluded by default # datastore_instances = false ## false by default for Datastores only # ## Datacenters datacenter_metric_include = [] ## if omitted or empty, all metrics are collected # datacenter_metric_exclude = [ "*" ] ## Datacenters are not collected by default. # datacenter_instances = false ## false by default for Datastores only # ## Plugin Settings ## separator character to use for measurement and field names (default: "_") # separator = "_" # ## number of objects to retreive per query for realtime resources (vms and hosts) ## set to 64 for vCenter 5.5 and 6.0 (default: 256) # max_query_objects = 256 # ## number of metrics to retreive per query for non-realtime resources (clusters and datastores) ## set to 64 for vCenter 5.5 and 6.0 (default: 256) # max_query_metrics = 256 # ## number of go routines to use for collection and discovery of objects and metrics # collect_concurrency = 1 # discover_concurrency = 1 # ## whether or not to force discovery of new objects on initial gather call before collecting metrics ## when true for large environments, this may cause errors for time elapsed while collecting metrics ## when false (default), the first collection cycle may result in no or limited metrics while objects are discovered # force_discover_on_init = false # ## the interval before (re)discovering objects subject to metrics collection (default: 300s) # object_discovery_interval = "300s" # ## timeout applies to any of the api request made to vcenter # timeout = "60s" # ## Optional SSL Config # ssl_ca = "/path/to/cafile" # ssl_cert = "/path/to/certfile" # ssl_key = "/path/to/keyfile" ## Use SSL but skip chain & host verification insecure_skip_verify = true —————————————————————————————————————
      • You only need to change the credential of vcenter and influxdb
      • Start and enable telegraf service after making the changes.
      • sudo systemctl restart telegraf
      • sudo systemctl enable telegraf

      Step: 3.1 Check InfluxDB Metrics

      • We need to confirm that our metrics are being pushed to InfluxDB and that we can see them.
      • If you are using authentication then open  InfluxDB shell like this:
      $ influx -username 'username' -password 'PASSWD'
      • We need to confirm that our metrics pushed to InfluxDB and that we can see them. If you are using authentication, then open the InfluxDB shell by this:
      $ influx
      • Then:
      > USE vmware
      • Using database vmware:
      • Check if there is an inflow of time series metrics.
      > SHOW MEASUREMENTSname: measurementsname—-cpudiskdiskiokernelmemprocessesswapsystemvsphere_cluster_clusterServicesvsphere_cluster_memvsphere_cluster_vmopvsphere_datacenter_vmopvsphere_datastore_datastorevsphere_datastore_diskvsphere_host_cpuvsphere_host_diskvsphere_host_memvsphere_host_netvsphere_host_powervsphere_host_storageAdaptervsphere_host_sysvsphere_vm_cpuvsphere_vm_memvsphere_vm_netvsphere_vm_powervsphere_vm_sysvsphere_vm_virtualDisk

      Step 4: Add InfluxDB Data Source to Grafana

      • Login to Grafana and add InfluxDB data source
      • Click on the configuration icon and then click datasource.
      • Click Add influxDB data source.
      • Insert all the relevant information under HTTP and influxDB details shown into the red boxes below:
      • If you used a password in your influxDB you might put it here.
      Grafana

      Step 5: Import Grafana Dashboards

      • The last action is to create or import Grafana dashboards:
      • Building a Grafana dashboard is a lengthy process, so we are using a community dashboard built by Jorge de la Cruz.
      Grafana
      • We will import this pre-build Grafana dashboard #8159. The moment you did import, you will see your Grafana dashboard.
      Grafana

      How Anisble Manage Configuration Files

      This article will discuss, where the Ansible configuration files are located and how Ansible selects them and how we can edit default settings.

      Configuring Ansible:

      The Ansible behavior can be customized by modifying settings in the Ansible configuration files. Ansible chooses its configuration file from one of many locations on the control node.

      •  /etc/ansible/ansible.cfg
        This file contains the base configuration of the Ansible. It is used if no other configuration file is found.
      • ~/.ansible.cfg
        This ~/.ansible.cfg configuration is used instead of the /etc/ansible/ansible.cfg because Ansible for .ansible.cfg in the home directory of the user.
      • ./ansible.cfg
        If the Ansible command is executed in the directory where the ansible.cfg is also present ./ansible.cfg will be used.

      Recommendations of Ansible configuration files:

      Ansible recommends creating a file in the directory from where you run the ansible command.

      Varibale ANSIBLE_CONFIG

      To define the location of the configuration file Ansible gives you a more handy option to define the configuration file by allowing you to change the environment variable named ASNIBLE_CONFIG. If you define this ANSIBLE_CONFIG variable, Ansible uses the configuration file that the variable specifies instead of any of the previously mentioned configuration file.

      Configuration File Precedence:

      Ansible Configuration File Precedence Table
      First preference Environment variable ANSIBLE_CONFIG overrides all other configuration files. If this variable is not settled, then second preference will be taken
      Second preferenceThe directory in which the ansible command was run is then checked for configuration file ‘ansible.cfg’. If this file is not present, then Ansible goes to third preference.
      Third PreferenceThe user’s home directory is checked for a .ansible.cfg file.
      fourth preferenceThe global /etc/ansible/ansible.cfg file is only used if no other configuration file is found.

       

      Due to Ansible’s capability to handle configuration from multiple locations, sometimes it makes the user confused to determine the active configuration file.

      So how use can determine which file is active?

      How to check which Ansible configuration file is being used?

      You can run the ansible –version command to identify which version of Ansible is installed and which configuration file is used.

      [ali@controller /]$ ansible --version
      ansible 2.9.16
      config file = /etc/ansible/ansible.cfg
      configured module search path = ['/home/ali/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
      ansible python module location = /usr/lib/python3.6/site-packages/ansible
      executable location = /usr/bin/ansible
      python version = 3.6.8 (default, Aug 24 2020, 17:57:11) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]
      [ali@controller /]$

      If you need servers to practice Ansible or Linux?

      SeiMaxim is a leading Dutch web hosting company and provides resources to learn Ansible and Linux. If you want to get virtual servers to learn Ansible you can place your order and use code SE-ANSIBLE211 to rent two servers in just 18 USD.

      All You Need To Know About iSCSI on VMware

      iSCSIBasics of iSCSI

      In the computing world, iSCSI is an acronym for Internet Small Computer Systems Interface, an (IP Internet Protocol)-based storage networking standard for linking data storage facilities. It provides block-level access to storage devices by carrying SCSI commands over a TCP/IP network.

      With the emergence of high-speed networks which includes 2.5 Gbps, 5 Gbps, 25 Gbps, 40 Gbps, 50 Gbps and 100 Gbps speeds iSCSI becoming more popular where Fiber channel still dominates in production environments but for the non-critical environment and the customers looking for a cheap storage solution iSCSI is the best option for them. In recent years vSphere has tons of improvement in the iSCSI software initiator especially with the jumbo frames support iSCSI is spreading widely in the industry.

      Let’s starts with the basics, it is one of the main IP storage standards especially for the none critical load. In the figure below a server is on network is accessing block storage.iSCSIType-1 hypervisors are capable to support different storage technologies and protocols for presenting external storage devices. We will discuss here mostly about VMware and a little bit about KVM. So vSphere is providing support for iSCSI storage since it’s the greatest version of all times “virtual infrastructure 3”.

      Adoption of iSCSI

      We do not need to create the new network as we do in FC, we can have SCSI in our common network for LAN, MAN & WAN.  TCP/IP have no limits on distance. Manpower and TCP/IP opensource tools are widely available, so these are the main benefits we have over FC if we implement iSCSI

      Nowadays you do not need a storage admin as companies were in need 10 years ago. Storage arrays evaluated a lot in recent years and it is now so easy to configure them. Management software will do all the raids and hardware monitoring for you. One of the main benefits of ISCSI implementation is it is inexpensive as compared to its other counterpart storage protocols such as fibre channel protocol.

      “The Bitterness of poor performance lasts long after the sweetness of a cheap price is forgotten” Michael Webster VMworld 2013

      When iSCSI use network interface cards rather than dedicated iSCSI adapters, interfaces expected to use significant amount of CPU resource of your servers. There are many ways to overcome this problem but one of them is to use TOE ( TCP Offload Engine) capable NIC.

      What TOE do?

      It just simply move the TCP packet processing tasks from server CPU to specialized TCP processor on the network adopter or it could be possible that move to the Storage device. The concept of offloading work from the main processor is similar to that governing graphics coprocessors, which offload 3D calculations and visual rendering tasks from the main CPU.

      The ability of TOE to perform full transport layer functionality is essential to obtaining tangible benefits. The important aspect of this layer is it being the process-to-process layer.

      In my point of view the cost is unquestionably the main issue that has hindered the adoption of TOE in general enterprise community. The normal TOE capable cards can range in price from $400 to $2000 and in some of the server you need to use the expansion slot or even raiser so additional cost and the benefit you get is not that big that everybody consider to buy TOE cards.

      Moreover, in my point of view over the time VMware has improved its vSphere a lot specially with the liberty to enable Jumbo Frames I would prefer to use the software iSCSI and on other side with TOE you don’t have it. For VMware please go to VMware HCL

      Difference between iSCSI and Fiber Channel

      One of the main difference between iSCSI and Fibre channel is the ways to handle the I/O congestion. So when an iSCSI path is overloaded or it drops packet and become substantially oversubscribed, this bad situation quickly grows and become worse. The performance further degrades because dropped packet must be resent, where as FC protocol is having a built in pause mechanism when congestion occurs. So both protocols are having different mechanism to handle congestion.

      Currently many vendors implemented delayed Ack and congestion avoidance as a part to there TCP/IP stack. VMware recommends consulting the iSCSI array vendor for specific recommendations around Delayed Ack.

      TCP delayed acknowledgment

      TCP delayed acknowledgment is a technique used to improve network performance. In essence, several ACK responses may be combined together into a single response, reducing protocol overhead.

      Difference between iSCSI and NAS

      NAS presents devices at the file level. NAS is specialized for serving files either by its hardware, software or configuration. Its often manufactured as a computer appliance.

      iSCSI Architecture

      iSCSI is an Internet Engineering Task Force (IETF) standard for encapsulating SCSI control and data in TCP/IP packets. In shown below figure you can see how iSCSI is encapsulated in TCP/IP and Ethernet frames.

      iSCSI

      VMware iSCSI Names

      Named globally unique and they are not bound to any ethernet adopters or IP addresses. iSCSI support two forms, one is Extended Unique Identifier (EUI) and iSCSI Qualified names IQN
      Basically, iSCSI is the client-server architecture. The clients of an iSCSI interface are known as initiators and the server that shares the storage area is known as targets.

      iSCSI Components

      There are two basic iSCSI components;

      Initiator

      It functions as an iSCSI client. An iSCSI initiator sends an SCSI command over an IP network. There are two kinds of initiators;

      A software initiator uses code to implement iSCSI, typically, this happens in the kernel device driver that uses the network card and network stack to emulate SCSI devices for a computer by speaking the iSCSI protocol.
      Nowadays all most all the popular operating system comes with the software initiators. In the table below you can find the dates when operating systems released their software intiators.

      Operating SystemFirst day releaseVersionFeatures
      VMware ESX2006ESX 3.0-7.XTarget,Multipath
      Linux20052.6.12,3.1Initiator,Target, Multipath,VAAI
      Windows200032000,WIN19Initiator,Target,Multipath
      FreeBSD20097.0initiator

      Target

      The iSCSI refers to a storage resource located on an iSCSI server as a target. These are typically data providers. This is your storage array and it provides distinct iSCSI targets for numerous clients.
      In the context of vSphere, iSCSI initiators fall into three distinct categories.

      Software iSCSI Adaptor

      This is VMware code built into the vmkernel, it enables your host to connect to the iSCSI storage device through the standard network adapter.

      Depended hardware iSCSI Adapter

      Provided by VMware this type of adapter can be a card that presents a standard network adapter and iSCSI offload functionality for the same port. An example of a dependent adapter is the iSCSI licensed Broadcom 5709.

      Independent hardware iSCSI Adapter

      This kind of adapter represents an independent hardware iSCSI adapter which is a card that presents either iSCSI offload functionality and standard nic functionality. The iSCSI offload functionality has independent configuration management that assigns the IP address, MAC address, and other parameters used for the iSCSI sessions. This is the kind we talked about earlier TOE cards. To identify if TOE and other TCP features are enabled or not, run the command:

      To check the status of TSO in ESX/ESXi, run the command:

      ethtool -k vmnicX

      To disable TSO within a Linux guest OS, run the command:

      ethtool -K ethX tso off

      Simplest Topology of An iSCSI Array

      In the figure below four ESX hosts are connected in the simplest form, so each ESX is having two uplinks and they are connected with two switches and on other side storage array is connected to the switch. All the connections are redundant.

      iSCSI

      Try to avoid vSphere NIC teaming and use port binding. with port binding you can utilise the multipathing for availability of access to the iSCSI targets.

      There will be some senarios where you need to use the teaming. If this is the case then turn off port security on the switch for the two ports on which the virtual IP address is shared. By turning off the security setting you can prevent spoofing of IP address.

      How to add an iSCSI initiator to the vSphere ESXi

      Before adding a new iSCSI initiator here are some recommendations.

      • Make sure that the host recognizes LUNs at start-up.
      • SCSI controller driver in the guest operating system should have a large queue. for Windows OS increase the value of the SCSI Timeout value parameter to tolerate delays I/O resulting from path failover.
      • Configure your environment to have only one VMFS datastore for each LUN.

      Select on the ESX hosts click configure and click +Add Software Adapter it will pop-up one more windows in the that windows chose “Add Software iSCSI Adopter.

      Read more