Lung are normally 3 centimeter or less

Lung Nodule Detection and
Classification: (Convolutional Neural Networks)


Arslan Hanif

of Science and Technology

of Management and Technology, Lahore

[email protected]


Lung nodules are commonly found in
radiographs and CT-scan images. Detection of lung nodules either they are
cancerous (Malignant) or non-cancerous (Benign) is a significant task to
perform. Malignant lung nodule detection in chest x-rays films is commonly performed
by Computer-Aided Detection (CAD) systems with help of radiologists.
Convolutional Neural Networks models are getting more mature for image
recognition and classification in many fields regarding image processing and
deep learning. Use of CNN for lung nodule detection and classification in chest
radiographs has no much explored. In this paper, I approached the lung nodule
detection and classification by CNN to increase sensitivity and accuracy. Resent
and Alex Net CNN models with transfer learning are used to detect and classify
nodules in chest radiographs. Both models are trained on a diverse set of
images. The proposed model is good in classifying radiographs as nodule or
non-nodule and it is also good at determining general nodule regions and locations.

1.     Introduction

Cancer is the most common cause of
death worldwide and Lung cancer is among the most significant types of cancer
1. As prevention is better than cure, therefore it is very necessary to detect
cancer as early as possible; detection methods could become more effective by
making them automated with help of computers. A lung nodule is a small round
spot type growth of tissue in the lung. Lung nodules are normally 3 centimeter
or less in size. Lung nodules that appear to be larger than 3 centimeter in
size have more possibility to be cancerous (malignant) than smaller ones.
Identification of malignant and benign (non-cancerous) nodules is very
necessary to detect cancer on early stage. Chest imaging has two major
techniques, X-ray imaging and computed tomography (CT). Radiographs of chest
show 1-dimentional view. Most common type of view in chest radiographs is posteranterior
view in which X-ray beam is passed from back to front of the chest. CT imaging
provides 3-dimensional view which is deduced using a rotational scanner which
takes X-ray images from many orientations. CT scans are better than radiographs
in order to provide a more complete view of chest internals which could help in
finding size, shape, density and location of nodules more easily. Despite all
these benefits of CT scan, there is a limitation of its availability and cost
which makes a barrier in data gathering from rural areas and smaller hospitals.
On the other hand, chest radiographs are lower in cost and also available in
almost every hospital. Radiographs have lower radiation effects than CT scans
on patients. These are the reasons for use of radiographs for detection of
chest problems.

Lung nodule detection was previously
performed by trained radiologists on the basis of circular objects found on
chest radiographs 2 which affect the results of nodule detection due to
variation in expertise of radiologists. Later Computer Aided Detection (CAD)
techniques were introduced and explored to reduce error in detection and consume
less time. Image processing technique was used to design algorithms for lung
nodule detection which find the locations of the chest radiographs having bright
objects of the defined size, shape and texture.

Convolutional neural networks have
been improved as time passed and researchers used CNN for image classification
and deep learning in multiple ways. CNN is also used to classify lung nodules
in some researches. But there is always a problem of using deep learning in
medical science; less amount of data relative to a field. Deep learning
networks are dependent on large set of data for learning. It is very difficult
to train a neural network on specific lung nodule images due to low amount of
data. Therefore, networks are first trained on large amount of data and then
used on images of lung nodules to detect and classify them. A similar work is
shown for better understanding 3.

In this paper, my work is to
classify the nodule as malignant or benign using Convolutional neural network
with transfer learning. Chest radiographs (both having nodules and non-nodule)
of type posteroanterior are used as input for a pre-trained CNN model. Two CNN
models are used for classification process and the output is in the form of
scores for input images (radiographs) having a malignant or benign nodule or no

2.     Related Work

Detection of lung nodules is still
performed by experienced radiologists majorly. Lung nodule detection is performed
on two major types of images; radiographs and CT scans. CT scans has advanced
the detection of nodules due to higher resolution and 3-dimensional view 4. Radiographs
provide 1-dimensional view and are less expensive with minimum radiology effect
on patients 5. Computer Aided System (CAD) is helping radiologists by
detecting nodules using highly-sensitive nodule detection pass. This high
sensitivity also causes false-positive results which are ultimately removed by
radiologists. Therefore, CAD systems which can reduce the false positives are
preferred. CAD systems which are in use are unable to find all nodules in the
chest; therefore there is still need of radiologists for expert scan to
remaining nodules 6.

A CAD system works with a sequence
of pre-defined procedures. These systems firstly performs segmentation using
pixel classification or shape models, secondly, detects potential regions of
candidate nodules using filtering and threshold. Thirdly, feature extraction is
performed on classified regions and then trained on a classifier, which outputs
suspiciousness rate for that region. After these steps, the outputs of regions
having greater rate are then presented to radiologists. This whole task is
firstly time consuming and secondly it is not utilizing the advancements of
computer science 6.

Convolutional Neural Networks not
used commonly for nodule detection or classification using X-ray images. Studies
show that small Convolutional neural networks having two or three layers were
used in 1995 and 1996 for classification of extracted regions to be cancerous
or non-cancerous 7 8. Researches mentioned above used pre-scanned CAD
systems for cropping out the candidate regions from X-ray images.

Two more studies show deep learning
Convolutional Neural Networks and transfer learning for classification of
images. Bush at el. used ResNet Trained on the large dataset of ImageNet to
detect and classify images as nodule, non-nodule and no nodule. They also
worked on localization of images. Pre-trained ResNet network model was used due
to small dataset of radiographs (only 256) 5. Bar et al. used AlexNet trained
on the same ImageNet dataset for detection of Right Pleural Effusion in X-ray
images 9. Roth et al. detected sclerotic spine metastases, lymph nodes, and
colonic polyps by training deep CNNs which increased sensitivity by 15 to 30% for
above mentioned tasks 10.

As CNNs have ability to classify
image by each pixel, which means similar type of spots or elements could be detected
on multiple places of same image, I will explore the ability of two
Convolutional Neural Networks models ResNet and AlexNet; to which extent they
are capable of detecting and classifying the nodules in radiographs without any
CAD system for cropping of potential nodule regions. CNNs will be self able to
directly identify the potential nodule regions, which is time efficient.

3.     Methods

For classification of radiographs,
two major Convolutional Neural Network models are used for classification of
images as images as nodule, non-nodule and no nodule. Models names are AlexNet
and ResNet, which are explained below.


AlexNet is a large, deep
Convolutional Neural Network, winner of 2012 ILSVRC. AlexNet model is a network
which consists 5 convolutional layers, max-pooling layers, dropout layers and 3
fully connected layers. AlexNet model which was originally designed to classify
image into 1000 possible categories, is modified in my case. I made changes in
the actual AlexNet model and restricted it to generate only 3 possible classes
(nodule, non-nodule and no nodule) of an image rather than 1000 classes; Fig. 1
is shown for better understanding of AlexNet model. This Network is trained on
ImageNet data, which consists of more than millions of images and thousands of
categories. ReLU (Rectified Linear Unit) is used for the nonlinearity functions
in AlexNet because of its faster performance in training of network. In
AlexNet, data augmentation techniques are used which have functionality of
image translation, patch extractions and horizontal reflections. The problem of
over fitting to the training data is solved by implementation of dropout

Fig: 1. Demonstration of AlexNet model with weights

First five layers of AlexNet network
are convolutional. Last three layers are fully connected. Convolutional layers
also include ReLu nonlinearity function between each layer. Output of this
model could thousands of classes extracted from input image but I have modified
it to generate only three classes which are required for this work only. This
modification was according to the requirement of the work to get desired


ResNet (residual network) models are
used for classification, in this work they are used with a slighter change; an
output of three classes is obtained from this network with scoring points of
each class to identify the resulting class. ResNet models are specially
designed for backward propagation of gradient results to improve training and are
trained on Imagenet dataset. Design of ResNet model is specially to reduce
training time. For this purpose, building blocks in ResNet are modified into bottleneck
design. Normally, residual functions are of two layers normally but in this
model 3 layers are used which are 1×1, 3×3 and 1×1 convolutions. 1×1 layers are
for dimensions restoration, which creates a bottleneck for 3×3 layer with low
amount of input/output dimensions. The 50-layer ResNet model is created by
replacing each 2-layer block with above defined 3-layer bottleneck block.
ResNet model contains 3.8 billion FLOPs 11.  ResNet model used for this work demonstration
is given in Fig.2.

Fig. 2: A
ResNet (50-layer) for classification of lung nodules into 3 classes with points
representing weight of each class. Required features are pulled out from 5
layers of the model and then passed from a fully connected layer (Pool5) to get
the required three class output (nodule, no nodule and non-nodule).

are processed through ResNet model. Pre-trained fixed weights are assigned to
the model.  Feature extraction process is
done on 5 layers and outputs of these layers are extracted with set of points
to identify the preference of each class. Pool1, res2c, res3d, res4f and pool5
layers are the feature extraction layers. A complete description of layers is
given in the Fig. 2 which shows that each output is generated after different
amount of convolution layers, ReLu, Batch Norm (BN) and pooling. Thus they
provide different output points from each feature. All extracted features are
then processed through a fully connected layer which will return three classes
as output as expected. The last fully connected layer is trained similar to the
Softmax classifier, which examines the outputs in the form of un-normalized log
probabilities of the classes (benign, malignant, non-nodule) and lowers the
cross-entropy loss rate among the labels of the output classes and actual image
labels.  After loss function, algorithm
tries to search the weights that cause minimizing of loss function.  Mini-batch gradient descent with momentum is
used for minimization.

training of all weights is accomplished saliency maps are created for indication
of regions which could have larger effect on the final classification.

4.     Dataset

Dataset used for lung nodule
detection could be CT-scan images or radiographs. I have used chest radiographs
obtained from Japanese Society of Radiological Technology (JSRT) 13. JSRT
dataset consists of total 246 grayscale radiograph images (Fig. 3), among which
154 are nodule and 93 are non-nodule. Additional information is also provided
along image which is visible on “ImagJ” tool for visualization. Researchers can
download database by simple registration on the website. All radiographs are
assured having nodules and non-nodules via CT scan and then separated by their
type. Images of this database have 2048 x 2048 resolution, 0.175mm pixel size
and (x, y)-coordinates of the lung.

Dataset is divided into 6 sets of 41
images. Each set includes a mixture of nodule and non-nodule images for better
training and testing. One set is separated for testing purpose. System is first
trained on a single set and then tested for verification and the process is
repeated on all sets for a diverse data training in order to get better
classification results.

AlexNet and ResNet models use RGB images
as input; JSRT images are converted into RGB from grayscale first and then used
for training and testing. AlexNet and ResNet models are used separately on this
dataset and the results are then cross-matched to identify the network which
performs better on chest radiographs. Caffe framework is used for the
implementation of this work. Caffe is used on linux. NVIDIA DIGITS is used for
better visualization of the experiments. Caffe framework needs NVIDIA GPU for
faster performance during training of the CNN models. Caffe needs CUDA 8 for
GPU working and python dependencies for proper working of framework.

Fig. 3: JSRT radiograph

5.     Results and Discussions

is concluded from the experiments that CNN models perform better on the
radiographs. CNN models trained on Imagenet which helped very much in training
of the networks and results showed that networks were well trained due to
diverse data provided by Imagenet. Test dataset was remained separate during
training in order to get real time results and to show actual performance of
the CNN models. This experiment tells that if CNN models come in practice then
they can improve the medical standards not only for cancer detection in chests
but also in many other aspects of medical science. CNN models reduce the need
of medical experts all the time; they can perform as a medical expert as they
become more trained with respect to increase in training data of patients.

6.     References

Media centre – cancer,

Lo, S.-L. Lou, J.-S. Lin, M. Freedman, M. Chien, and S. Mun. Artificial
convolution neural network techniques and applications for lung nodule
detection. IEEE Transactions on Medical Imaging, 14(4), 1995.

S. Razavian, H. Azizpour, J. Sullivan, and S. Carlsson. Cnn features
off-the-shelf: an astounding baseline for recognition. arXiv:1403.6382, 2014.

Sone and et al. Characteristics of small lung cancers invisible on conventional
chest      radiography and detected by
population based screening using spiral ct. The British Journal of Radiology,

 I. Bush, Lung Nodule detection and
classification, Stanford Computer Science, 2016.

van Genneken, C. Schaefer-Prokop, and M. Prokop. Compter-aided diagnosis: How
to move from the laboratory to the clinic. Radiology, 261(3), 2011.

Lo, S.-L. Lou, J.-S. Lin, M. Freedman, M. Chien, and S. Mun. Artificial
convolution neural network techniques and applications for lung nodule
detection. IEEE Transactions on Medical Imaging, 14(4), 1995.

Lin, S.-C. Lo, A. Hasegawa, M. Freedman, and S. Mun. Reduction of false
positives in lung nodule detection using a two-level neural classification.
IEEE Transactions on Medical Imaging, 15(2):206–217, 1996.

Bar, I. Diamant, L. Wolf, and H. Greenspan. Deep learning with non-medical
training used for chest pathology identification. SPIE, 9414, 2015.

R. Roth and et al. Improving computer-aided detection using convolutional
neural networks and random view aggregation. IEEE Trans. on Medical Imaging,

He, X Zhang, S Ren, J Sun. Deep Residual Learning for Image Recognition. Arxiv:
1512.03385v1 cs.CV. 2015.

Cancer Society. Key statistics for lung cancer, 2016.       

J, K. S, I. J, M. T, K. T, K. K, M. M, F. H, K. Y, and D. K. Development of a
digital image database for chest radiographs with and without a lung nodule:
Receiver operating characteristic analysis of radiologists’ detection of
pulmonary nodules. AJR, 174:71–74, 2000.