Checking ports with telnet or netcat

 Recently I was working on AWS and was deploying applications on a Fargate cluster. I wanted to make sure the application was up and running and the ports were accessible by other applications. For that purpose I created a bastion host (EC2) but because of the restrictions on my role, I could not install telnet or nc on that EC2 instance. 

But I was able to connect to the ports using the below bash command

</dev/tcp/10.191.226.250/13133 && echo Port is open || echo Port is closed

Read More

How I solved "Expected tensor for argument #1 ‘indices’ to have scalar type Long" error

Today I was trying to train my 2 layer LSTM on some long documentry extract and kept geting this irritating error
Expected tensor for argument #1 ‘indices’ to have scalar type Long; but got CPUFloatTensor instead (while checking arguments for embedding)

After many google searches and stackoverflow crawling, I figured out that the error was caused by the following line of code

dataset = TensorDataset(torch.Tensor(features), torch.Tensor(target))

Changed it to

dataset = TensorDataset(torch.tensor(features), torch.tensor(target))

and it fixed the issue

Read More

Resolve the issue 'cannot import name multivariate_normal'

The multivariate_normal component in the scipy stats library is a very useful component for many machine learning applications. I faced an issue while trying to import this library in my Jupyter notebook and I wanted to share how I fixed it. I am running python 2.7 on an Ubuntu 14.04 machine.

Read more »

Read More

Solving 0.0.0.0 permission denied (publickey) error message on Hadoop name node

For a Hadoop cluster to work properly, all the hadoop components like NameNode, SecondaryNameNode, JobTracker and ProcessManager must be started. If you happen to create a hadoop cluster by yourself like I mentioned in my previous blog, when you start the name node by using 'start-dfs.sh', you might get the below error

'0.0.0.0 permission denied (publickey)'

This is due to the fact that, by default, the hadoop core process will try to start secondary namenode on the default ip address 0.0.0.0 which by default should be pointing to localhost. In order to do that it tries to establish an SSH connection to the ip address 0.0.0.0 and as the error message states, it fails to make the connection because the passwordless SSH is not setup correctly for the host 0.0.0.0. Below steps will help you fix this

1) If you are using a config file for passwordless ssh (mostly in case of AWS servers)

In order to setup passwordless ssh, you might have created a config file in .ssh folder of your home directory with the entries for hostname, userid and the keyfile to be used while doing ssh, as shown below

Read more »

Read More

Solving jps: command not found error message

jps is a command line utility that comes with jdk which allows you to view all the Java processes running on a host. It is extremely useful when your environment has multiple background java processes running, like it happens in a Hadoop cluster. If you happen to create a hadoop cluster by yourself as mentioned in my previous blogs, 'jps' command is very useful to diagnostic utility to make sure the NameNodes, DataNodes, ProcessManager and ResourceManager processes are running.

Sometimes if you try to use 'jps' and get the error message "jps: command not found error message" here is how you solve it

Read more »

Read More

Setting up a Hadoop Cluster in RHEL 6 - Preparing the servers

In this two part series, we will discuss how to setup a Hadoop cluster on RHEL 6. I was inspired by my friend who did the Big Data Specialization course of University of California San Diego through Coursera and I thought it will be a good learning experience for me to setup a Hadoop cluster myself as well. Even though Cloudera's open source paltform is the most common distribution of Hadoop, I really wanted to understand some low-level details of Hadoop components and how they interact with each other. There fore, I thought it will be useful to install and configure the basic Apache Hadoop distribution from scratch.

In this part, we will discuss the setup of  Linux environment and how to install the Hadoop distribution. In Part 2 we will discuss how to configure the core components of the Hadoop ecosystem and start the server. Okay, lets get started

Read more »

Read More

Install and configure PuTTY for making SSH connections

Being able to securely connect to remote hosts from windows using SSH is a very important and common requirement, especially when you are working with remote linux servers. SSH is a high-security protocol. It uses strong encryption to shield your connection from eavesdropping. capture, and other sorts of attack. In this post we will discuss exactly how to do this using PuTTY tool. We will also show how to create and save PuTTY sessions and discuss how to configure login without password using key pair.

PuTTY is a free, open source SSH client and a terminal emulator for windows. It has a very easy to use user interface and the ability to configure and store sessions, so that we can connect to configured hosts in a single click.

Read more »

Read More

Resolving Unsatisfied link error in OpenNI java on Windows

If you are getting any of the following error while trying to Run OpenNI on a windows PC, follow the below steps to rid of the errors (I am assuming you are using Eclipse)

java.lang.UnsatisfiedLinkError: C:\Program Files\OpenNI2\Redist\OpenNI2.jni.dll: Can't find dependent libraries

or

java.lang.UnsatisfiedLinkError: no OpenNI2.jni in java.library.path

Read more »

Read More

Connecting MK808 device to ADB using the bridge and OEM USB Drivers

MK808 is a very cheap and handy Android device to play around. Connected to an HDMI monitor, we can easily run Android applications on this device. Before starting, we need to first connect MK808 to the ADB so that we can install/run/debug applications. OEM drivers provided by Google is not sufficient to connect the MK808 device to the ADB. After installing the OEM drivers, please follow the below steps

Read more »

Read More