Big Data for Security Analytics

Workshop on Big Data for Security Analytics


Almost all of the major breaches in the past have shown that the SOC/SIEM did not do their job well and alert the client to the breach. What are SOC/SIEMs missing and why? And how do fix this problem? The search for the proverbial needle in the haystack has become a daunting task as the haystack is now phenomenally big. How can Big Data help determine security intelligence?

This training workshop will explore addressing this problem using big data analytics and leveraging Hadoop, ElasticSearch, Logstash, Kibana, Beats along with a layer of Machine Learning on top of this data to determine active compromises in your network, possible frauds leads and visualize all this information for better perception by the human eye. As a completely hands-on workshop participants will be taken through the steps of implementing ELK and Hadoop as well as installing the Apache Metron project that is built on the same principles. Logs from real-world scenarios will be shared and analytics run on these to see the real value that a Big Data setup can deliver.


K. K. Mookhey

KK is one of the pioneers of information security in India. Having begun his firm as a one-man show in 2001, it has now grown to a team of over 250 consultants spread across multiple locations in India and the Middle East. He is a trusted consultant and trainer to organizations all across the globe on various aspects of information security. He is well-versed with the security challenges of various industry verticals, and also with international standards and frameworks such as ISO 27001, PCI DSS, COBIT, HIPAA, etc.

He is the author of two books (on Linux Security and on the Metasploit Framework) and of numerous articles on information security. He was the first security researcher from India to present at Blackhat in 2004 (on ‘Detection and Evasion of Web Application Attacks’) and since then has spoken at numerous conferences such as Interop, OWASP, NullCon, etc. He is currently overseeing the research activities within NII focused on use of big data in security, building various automation solutions, and security impact of the Internet of Things.

Wasim Halani

Wasim is one of the senior most consultants at NII. He started as a fresher about 8 years back and since then has been involved in various technical assessments in different industries and business verticals within India and internationally. He is currently serves as the Head of Innovations and Research (InR) team at NII, where he is responsible for introducing new ideas, tools and vectors for the Security Assessment practice. He also works to introduce new service models that NII can provide to it’s clients.

As part of his current research, he is leading a team to overcome limitations within existing security monitoring solutions by exploiting advancements in Big Data, Analytics and Machine Learning, to improve threat intelligence and monitoring and enabling early detection of advance threat actors.

Wasim is also actively involved in the Info-Sec community in India. He leads the NULL chapter in Mumbai and has participated in conferences like OWASP, SecurityByte, and Malcon.


Date: 26th – 28th September, 2016

Venue: Mumbai

The workshop is split into two parts:

  • Day 1: Introduction to Big Data technologies and Security Analytics
  • Day 2 & 3: Hands-on workshop using Big Data for Security

Who should attend

  • Day 1 is primarily intended for everyone who wish to understand and explore the Big Data ecosystem and how it can be used for Information Security Analytics and Visualization. This session does not have too much hands-on. Participants will be asked to run visualization exercises on an existing Kibana instance. However, a basic understanding of security monitoring and log analysis is required.
  • Day 2 & Day 3 involves hands-on workshop on using Big Data technologies for security analytics and visualization. Though managers are welcome to attend, the sessions are targeted at those who would want to implement such a setup and have experience in security monitoring and/or security visualization. Those interested in attending the hands-on workshop, should necessarily participate in the Day 1 session for introduction to Big Data.

Detailed Schedule

Session 1:
  • Introduction to Big Data
    • Generic use cases
    • What it is? What it is not?
Session 2:
  • Demystification of jargon and products
    • Hadoop
      • MapReduce
      • YARN
      • HDFS
    • Spark
    • Pig
    • Storm
    • ELK
      • ElasticSearch
      • Logstash
      • Kibana
      • Beats
Session 3:
  • ELK quick install and demo
    • Vulnerability Dashboard
Session 4:
  • Introduction
    • Security Visualization & Analytics
    • Active Threat Hunting
    • Machine Learning
Session 5:
  • Hadoop fundamentals
Session 6:
  • Hadoop installation and small demo
Session 7:
  • ELK Concepts
    • Index
    • Documents
    • Shards
    • Filters
    • Query
    • Aggregations
  • ELK Hands-on
    • Full Installation steps on Linux
    • Creating basic logstash file
    • Grok Patterns
      • Apache log
      • Firewall
    • Geo-IP
Session 8:
  • ELK Hands-on (contd…
    • Windows Logon Events Monitoring
      • Top logon fails
      • Single user logon to multiple systems
Session 9:
  • Active Threat Hunting
    • Concepts and Fundamentals
      • Why traditional SIEM fails
      • Why big-data
    • Threat-Intel Translation configuration
    • ELK based demo
  • Ingesting logs using Python
    • Creating VM dashboards
  • Log Metrics using ELK
Session 10:
  • Security Visualization Concepts
    • Outliers
    • Good vs Bad visualization
  • Lab exercises
    • Finding anomaly using Kibana Visualization
Session 11:
  • Machine Learning
    • Code development theory
    • Demo use-case
Session 12:
  • Apache Metron introduction
    • Background
    • Features
    • Demo

Ethical Hacking Training

This course goes deep down into depths of networking, systems, web applications, actual exploitation & helps beginners to take their confident first step towards information security field.

  • 6 Weeks Comprehensive Training
  • Built by Experienced Professionals
  • Regularly update on tools, techniques in course content