Thursday, January 20, 2022

Kubernetes Troubleshooting

Kubernetes Troubleshooting

View Console logs of a Container (stdout)

kubectl logs pod flaskpod-6db57ff68b-ckklj



Tail Console Logs of a Container

kubectl logs --follow flaskpod-557d6745c7-qkd5v




View recent logs (events) of a container

Kubectl describe po/nginx-7f4558cf8-6jjg2

Kubectl describe pod pod-name

Shows at the bottom the recent events


Or


Kubectl logs myPod -previous



Execute a command within a container

kubectl exec -it pod/mysql-7d8f78bf86-clcg5 /bin/bash


Tip: sometimes not all bash commands are available within a container, but can sometimes be very useful - e.g. if it is a MySql container you can run mysql -ppassword to run SQL commands (where password = password).



Launch busybox in a Namespace

Yaml for busybox: 

apiVersion: v1
kind: Pod
metadata:
name: busybox
namespace: default
spec:
containers:
- image: busybox
command:
- sleep
- "3600"
imagePullPolicy: IfNotPresent
name: busybox
restartPolicy: Always

kubectl apply -f jumpod.yml

pod/busybox created


kubectl exec -it busybox /bin/sh



View detailed deployment info

Kubectl get deploy -o wide



View the rollout history of a deployment

Kubectl rollout history deployment blah 



Get pods in all Namespaces

Kubectl get pod -A

Is get in all namespaces



Apply a deployment file

Kubectl apply -f deployment.yaml 


Create a namespace

kubectl create namespace monitoring





Terraform 101

 Terraform 101

Terraform is used for Infrastructure as Code (IaC).

Terraform is Declarative, NOT Imperative, and defines the DESIRED STATE.

What does that mean? For initial infrastructure creation, they will be very similar (e.g. both will say create 5 servers), but when you update your infrastructure there are differences.

For example, say you have 10 servers, and you want to remove 3...

Imperative commands to update your infrastructure would look like:

- remove 3 servers

Declarative commands to update your infrastructure would look like:

- have 7 servers


Terraform will work out what it needs to do to achieve the Desired state.

With Terraform, you re-run the whole configuration file to achieve your update.

Terraform Stages

refresh - query infra to get current state

plan - create an execution plan (plan to achieve the desired state, based on the current state). No changes.

apply - execute the plan

destroy - removes whole setup, removing elements in the correct order. E.g. remove servers before remove VPC. Destroy also runs a Refresh, and then a Plan before removing the resources.

Terraform plugin for VSCode

https://marketplace.visualstudio.com/items?itemName=HashiCorp.terraform


Install Terraform on Mac

First install Homebrew - https://brew.sh/

https://learn.hashicorp.com/tutorials/terraform/install-cli

brew tap hashicorp/tap

brew install hashicorp/tap/terraform

brew update

brew upgrade hashicorp/tap/terraform


Terraform Basic Operation

In vscode, create a new directory - e.g. Documents\terraform

create the file: main.tf

provider "aws" {
region = "us-east-1"
access_key = "access-key"
secret_key = "your-secret-key"
}

resource "aws_vpc" "myvpc" {
cidr_block = "10.0.0.0/16"
}


This will create a VPC resource - for syntax see:

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc


In VSCode, click the 'View' menu and select "Terminal".

Now, run the command:

terraform init



Initializing the backend... Initializing provider plugins... - Reusing previous version of hashicorp/aws from the dependency lock file - Using previously-installed hashicorp/aws v3.72.0 Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work. If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary.

Terraform has successfully initialised the plugin for AWS.

terraform plan



The terraform plan command examines the current state, and works out what needs to be changed in order to achieve the desired state.

In the main.tf file we specified a VPC with CIDR 10.0.0.0/16, and here terraform can see that it currently doesn't exist - hence the (known after apply) lines.
For example the arn (amazon resource name) for the VPC is unknown because we haven't created it yet.


Terraform Apply


Again, the apply command examines the current state, and says what it will do in order to achieve the desired state.
Type 'yes' to continue to perform the actions.




Terraform Destroy


This will destroy the resources listed in your main.tf file.


Terraform will examine the current state, and will say what actions it will take to destroy the resources.
Type 'yes' to continue.




State File

The state file is called 'terraform.tfstate'. This contains the current state.
There is also a back file called 'terraform.tfstate.backup', but this isn't the latest.

When you run an plan, apply, or destroy command, the terraform.tfstate file will be updated. The old contents of the file will be copied to the backup file.

Variables

Defined in the main.tf file.

Variables are a way of setting a value which can be used multiple times.
Variables can be a string, number, boolean, list, map, tuple, or object.

String Variables











Number Variables










Boolean Variables








List Variables

Lists can contain strings, numbers, or boolean values.







Note for lists, like an array, the first element is accessed by item number 0.
The value of item 0 = 'Value1'

Map Variables

Maps are key-value pairs.

The elements of a map are accessed using the key you gave,
For example:














When you access key1, its value is 'Value1'.

Tuples

Tuples are similar to Lists, but can contain different data types.

variable "mytuple" {
type = tuple([string, number, string])
default = ["cat",1,"dog"]
}


Objects

variable "myobject"{
type = object({name = string, port = list(number)})
default = {
name = "fw"
port = [22,25,80]
}
}


Using Variables

Variables are referenced using 'var.'
variable "vpcname" {
type = string
default = "myvpc"
}

resource "aws_vpc" "myvpc" {
cidr_block = "10.0.0.0/16"
tags = {
Name = var.vpcname
}
}


Or to access a list variable:
variable "mylist" {
type = list(string)
default = [ "Value1","Value2" ]
}

resource "aws_vpc" "myvpc" {
cidr_block = "10.0.0.0/16"
tags = {
Name = var.mylist[0]
}
}


Or to access a map variable:
variable "mymap" {
type = map
default = {
key1 = "Value1"
key2 = "Value2"
}
}

resource "aws_vpc" "myvpc" {
cidr_block = "10.0.0.0/16"
tags = {
Name = var.mymap["key1"]
}
}

Note, the key name is in quotes!


Input Variables

Allows user to manually specify a value when running terraform plan
variable "inputname" {
type = string
description = "set the name of the vpc"
}

resource "aws_vpc" "myvpc" {
cidr_block = "10.0.0.0/16"
tags = {
Name = var.inputname
}
}

Now when running terraform plan, you will see:







Outputs

After Terraform creates a resource, you may want to know some information about it such as the arn or id. Note, this will only work with terraform apply.

To see which attributes can be fetched, see the Terraform documentation, e.g gor AWS VPC resource, the attributes are:  https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc

resource "aws_vpc" "myvpc" {
cidr_block = "10.0.0.0/16"
tags = {
Name = var.inputname
}
}
output "vpcid" {
value = aws_vpc.myvpc.id
}
output "vpcarn" {
value = aws_vpc.myvpc.arn
}







Note: the outputs get saved to the State file terraform.tfstate


Creating an EC2 instance

The documentation for EC2 instance is 'aws_instance'


main.tf
provider "aws" {
region = "us-east-1"
}

resource "aws_instance" "ec2instance" {
ami = "ami-08e4e35cccc6189f4"
instance_type = "t2.micro"
}

output "ec2_arn" {
value = aws_instance.ec2instance.arn
}














Docker 101

 Docker 101

Install Docker for Mac

https://docs.docker.com/get-docker/

Create a Container

To create a container in docker, you need to first create a folder.
Within this folder you will need 3 docker files, and the files (code) to run your app.
The docker files are:
  • docker-compose.yaml
  • requirements.txt
  • Dockerfile

The following example here is a flask container, which uses a python script (app.py).
We have 4 files, all located in the same directory. The name of the directory doesn't matter.

app.py
Dockerfile
requirements.txt
docker-compose.yaml

Dockerfile

FROM python:3.8

WORKDIR /app
COPY . .

RUN pip install -r requirements.txt

ENTRYPOINT ["python"]
CMD ["app.py"]

The first thing here is the image which is used as the base image - denoted by the line "FROM python:3.8". Phyton 3.8 is a standard image containing Python. We then add to this image to install additional Python libraries before the program is executed - in this case 'pip install -r requirements.txt'.
Finally the 'Dockerfile' contains what should be executed when the container loads (python app.py). 

requirements.txt

Flask==0.12
flask-restful==0.3.5
flask-mysqldb==0.2.0
mysqlclient==1.4.2.post1

The requirements.txt file simply includes the python libraries which should be installed before the program is executed. This is the same as you would run on your PC - e.g pip install Flask==0.12.

docker-compose.yaml

version: "3.7"

services:
helloworld:
build:
context: ./
ports:
- 5000:5000

The docker-compose.yaml file states how the network requirements for the application - how it should be exposed to the outside world.


We now run the command:
docker compose build
from the directory which includes the 4 files mentioned above.

This will build the container, first downloading the image for python (python:3.8), and then installing the additional python libraries mentioned in the requirements.txt file. The docker image will include all of the files in the directory from where you ran the 'docker compose build' command.

The docker image will be automatically imported to docker running locally on your PC. To view it, run:
docker images

The name of the image = the name of the folder from where you ran the 'docker compose build' command concatenated with and underscore and the name of the service from the docker-compose.yaml file. In this case, my folder name is 'test' and the service name is 'helloworld'. The image name is therefore 'test_helloworld'.

Next, you want to upload the image to docker hub. To do that you first need to setup an account (free) on dockerhub https://hub.docker.com.

In order to upload the image to docker hub you must first login with a browser and create a repository on https://hub.docker.com. Make a note of the name of your repository. You can create a public or private repository. For this example I used a public repository. 

Next, you need to tag your image with your dockerhub username.

If your username is 'my-username', the repository name is 'my-repo', and the image created by 'docker compose build' is named 'testhelloworld', the tag command would be:

docker tag test_helloworld my-username/myrepo:aUniqueTagName

Note 'aUniqueTagName' is used to identify the image you are uploading and will be used in pull requests (downloads) against docker hub. The ':aUniqueTagName' is optional for the docker tag command, but I strongly recommend using it!

If you run docker images again, you will see your tagged image there.

Before uploading the image to docker hub, you must login to docker hub on the command line. To do that, run:


docker login -u your-username -p your-password docker.io


You can now upload your image to docker hub using:


docker push my-username/myrepo:aUniqueTagName



If you are using the image for a kubernetes deployment, in your deployment.yaml file, the image would be referenced as:
spec:
containers:
- name: name-of-your-container
image: docker.io/my-username/myrepo:aUniqueTagName




Other Docker commands

show running containers

docker ps 

show running and stopped containers

docker ps -a













Monday, September 06, 2021

Windows NTFS Apply New Permissions to SubFolders and Files

 icacls D:\MyPath /grant:r "domain\account":(OI)(CI)M /T


Run this from PowerShell:

cmd.exe /c  "icacls D:\MyPath /grant:r 'domain\account':(OI)(CI)M /T" >> $logpath




Monday, July 05, 2021

Azure list Bastion hosts

az network bastion list

Install Azure CLI for Mac

 1) Install Homebrew

https://brew.sh

2) Before doing a 'brew update' you have to run the following commands:

To `brew update`, first run:

  git -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core fetch --unshallow

  git -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask fetch --unshallow


3) Install Azure CLI
https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-macos
brew update && brew install azure-cli

4) Login to Azure
az login


Creating an Ansible Control Server for Windows Targets

Creating an Ansible Control Server for Windows Targets

OS: Ubuntu 16.04 Server on Azure

In order for the Control server to use Windows targets, you need Ansible v2.9 or later.

To get Ansible v2.9, install Ansible using pip3 (apt-get installs an older version of Ansible).

To install Python:

sudo apt install sofware-properties-common

sudo add-apt-repository ppa:deadsnakes/ppa


sudo apt update


sudo apt install python3.8


sudo apt install python3-pip


To install Ansible:

pip3 install ansible


Get ansible version:

ansible --version


Install pywinrm (in order to communicate with Windows targets)

pip3 install "pywinrm>=0.3.0"


Ansible Windows Modules:

win_shell

win_command

win_msi

win_copy


Ansible files required on control host at a minumum

e.g. Inside the folder: /home/azureuser/ansible

2 files: 'hosts' and 'ansible.cfg'

ansible.cfg file - specifies where hosts file is.

hosts (not for prod - passwords are cleartext & no cert validation)

[win]

10.2.0.5


[win:vars]

ansible_user=userWithAdminRightsOnTarget

ansible_password=password

ansible_connection=winrm

ansible_winrm_server_cert_validation=ignore


ansible.cfg

[defaults]

inventory=hosts




Example Playbook

---

- hosts: win

  tasks: 

  - name: Copy File

    win_copy: 

      src: C:\temp\1.txt

      dest: c:\temp2\

      remote_src: yes




Friday, October 30, 2020

Kerberos Test Page (ASP.NET)

 Its old, but this still works and is super useful:

https://docs.microsoft.com/en-us/archive/blogs/friis/asp-net-authentication-test-page

zip file at the bottom of the page to download the files.


Tested successfully on Server 2016. Make sure the app pool is set to 'Classic .NET AppPool' (in Advanced settings for the website).

Also set the normal Authentication settings for a Kerberos site:

- Anonymous Authentication = Disabled

- ASP.NET Impersonation = Enabled

- Windows Authentication = Enabled (and in Providers Negotiate is highest)

<%@ Page Language="C#" %>

<script runat="server">


 protected void dumpObject(object o, Table outputTable)

    {


    try

{

        Type refl_WindowsIdenty = typeof(System.Security.Principal.WindowsIdentity);

        

        System.Reflection.MemberInfo[] refl_WindowsIdenty_members = o.GetType().FindMembers(

                System.Reflection.MemberTypes.Property,

                System.Reflection.BindingFlags.Public | System.Reflection.BindingFlags.Instance,

                delegate(System.Reflection.MemberInfo objMemberInfo, Object objSearch) { return true; },

                null);


        foreach (System.Reflection.MemberInfo currentMemberInfo in refl_WindowsIdenty_members)

        {

            TableRow r = new TableRow();

            TableCell k = new TableCell();

            TableCell v = new TableCell();

            System.Reflection.MethodInfo getAccessorInfo = ((System.Reflection.PropertyInfo)currentMemberInfo).GetGetMethod();


            k.Text = currentMemberInfo.Name;

            object value = getAccessorInfo.Invoke(o, null);

            if (typeof(IEnumerable).IsInstanceOfType(value) && !typeof(string).IsInstanceOfType(value))

            {

                foreach (object item in (IEnumerable)value)

                {

                    v.Text += item.ToString() + "<br />";

                }

            }

            else

            {

                v.Text = value.ToString();

            }


            r.Cells.AddRange(new TableCell[] { k, v });

            outputTable.Rows.Add(r);

        }

}

    catch

{}


    }


protected void Page_Load(object sender, EventArgs e)

    {


    AuthMethod.Text = "Anonymous";

    AuthUser.Text = "none";



   //***** Authentication header type (Basic, Negotiate...etc)


   string AUTH_TYPE=Request.ServerVariables["AUTH_TYPE"];

   if (AUTH_TYPE.Length>0) AuthMethod.Text = AUTH_TYPE;


   //***** Authenticated user

   string AUTH_USER=Request.ServerVariables["AUTH_USER"];

   if (AUTH_USER.Length>0) AuthUser.Text  = AUTH_USER;

   

   //***** If NEGOTIATE is used, assume KERBEROS is length of auth. header exceeds 1000 bytes


   if (AuthMethod.Text == "Negotiate")

{

string auth=Request.ServerVariables.Get("HTTP_AUTHORIZATION");

if ((auth!=null)&&(auth.Length > 1000))

AuthMethod.Text = AuthMethod.Text + " (KERBEROS)";

else 

AuthMethod.Text = AuthMethod.Text + " (NTLM)" ;

}



ThreadId.Text=System.Security.Principal.WindowsIdentity.GetCurrent().Name;


        //set the process identity in the corresponding label

        dumpObject(System.Security.Principal.WindowsIdentity.GetCurrent(), tblProcessIdentity);

        

        //set the thread identity in the corresponding lable

        dumpObject(System.Threading.Thread.CurrentPrincipal.Identity, tblThreadIdentity);


int loop1, loop2;

NameValueCollection coll;


// Load ServerVariable collection into NameValueCollection object.

coll=Request.ServerVariables; 

// Get names of all keys into a string array. 

String[] arr1 = coll.AllKeys; 

for (loop1 = 0; loop1 < arr1.Length; loop1++) 

{

TableRow r = new TableRow();

        TableCell k = new TableCell();

                TableCell v = new TableCell();

k.Text=arr1[loop1];

     String[] arr2=coll.GetValues(arr1[loop1]);


for (loop2 = 0; loop2 < arr2.Length; loop2++) {

      v.Text=v.Text+Server.HtmlEncode(arr2[loop2]);

  }



  v.Text=Server.HtmlEncode(arr2[0]);

r.Cells.AddRange(new TableCell[] { k, v });

tblSrvVar.Rows.Add(r);

}


/*

IEnumerator en = Request.ServerVariables.Keys.GetEnumerator();

en.MoveNext(); 


foreach (string key in Request.ServerVariables.Keys) 

{ 

TableRow r = new TableRow();

        TableCell k = new TableCell();

                TableCell v = new TableCell();

k.Text = key;

v.Text=Request.ServerVariables[key];

r.Cells.AddRange(new TableCell[] { k, v });

tblSrvVar.Rows.Add(r);

}*/


    }

    

</script>


<html>

<body>

<font face="Verdana">

<center><H2>WHO Page</H2><br>

<table border=1>

<tr>

<td><img src="img1.gif"></td>

<td><img src="img2.gif"></td>

<td><img src="img3.gif"></td>

<td><img src="img4.gif"></td>

<td><img src="img5.gif"></td>

<td><img src="img6.gif"></td>

<td><img src="img7.gif"></td>

<td><img src="img8.gif"></td>

<td><img src="img9.gif"></td>

<td><img src="img10.gif"></td>

</tr>

</table>

</center>

<table border=1>

<tr><td>Authentication Method </td><td><b><asp:label id=AuthMethod runat=server/></b></td><td>Request.ServerVariables("AUTH_TYPE")</td></tr>

<tr><td>Identity </td><td><b><asp:label id=AuthUser runat=server /></b></td><td>Request.ServerVariables("AUTH_USER") or System.Threading.Thread.CurrentPrincipal.Identity</td></tr>

<tr><td>Windows identity </td><td> <b><asp:label id=ThreadId runat=server /></b></td><td>System.Security.Principal.WindowsIdentity.Getcurrent</td></tr>

</table>


<fieldset>

            <label>Identity (System.Threading.Thread.CurrentPrincipal.Identity)</label>

            <asp:Table ID="tblThreadIdentity" runat="server"></asp:Table>

        </fieldset>



<fieldset>

            <label>Windows Identity (System.Security.Principal.WindowsIdentity.GetCurrent)</label>

            <asp:Table ID="tblProcessIdentity" runat="server"></asp:Table>

        </fieldset>


Dump of server variables :

<br><br>

<asp:Table ID="tblSrvVar" runat="server"></asp:Table>


<form name="form1" method="POST" action="default.aspx">

Sample Form<br>

<input NAME="sometext" TYPE="Text" SIZE="24">

<input TYPE="submit" VALUE="Submit" NAME="SubmitForm" > 

</form>


</font>

</body>

</html>