From the big guys such as Jenkins and Visual Studio to open-source tools including GoCD and BuildBot, surely one of these automation tools can work for. 21 Automated Deployment Tools You Should. QTP is a great all-purpose test automation tool. I do not know much about Siebel Retail 8.1 GUI - is it web-based or thick client? If web-based, there are a ton of alternatives, including open source. If it's a thick client, you are limited in your choices. Cisco Network Automation Tools. Aside from all the open source options, Cisco are not lagging behind and have their own suite of Cisco network automation tools. For a long time there has always been Cisco Prime Infrastructure which does provide a way to automatically backup configs and push config via templates.

• Open Source Office Automation
• Open Source Automation Tools For Siebel Logos
• Open Source Automation Server

One of the simplest ways to tell whether a data center is being well managed is to see if the individual servers have pet names. Naming a server suggests the machine has some characteristics that need to be uniquely addressed—which, more often than not, means relying on manual processes that wind up making the cost of IT labor prohibitive.

“Anytime you see servers named after characters from ‘Lord of the Rings,’ it’s usually a sign of a complex, messy environment,” said Brent Jones, senior systems architect for Smarsh, a provider of an archiving services in the cloud for electronic communications. “When you look at advanced organizations such as Facebook or Google that we all try to emulate, you really don’t see machines that have individual names.”

Smarsh is part of a growing legion of IT organizations that rely on open-source IT automation tools such as Puppet from Puppet Labs or Chef from Opscode to automate management functions that previously required a lot of manual intervention on the part of an administrator.

While the concept of IT automation has been around since the early days of the mainframe, it’s only with the availability of easily accessible open-source automation tools that more IT professionals have begun applying these concepts to the management of distributed systems.

As is often the case in IT, necessity has become the mother of invention. With the advent of virtualization, the number of virtual servers running on top of physical servers has exploded. At the same time, the number of storage systems has increased dramatically, along with the sheer volume of data needing to be managed. Throwing additional administrators at this problem not only doesn’t scale, it’s simply cost-prohibitive. More challenging yet is the fact that, in the age of the cloud, IT administrators are now being asked to manage distributed system resources across multiple data centers.

To solve this issue, VMware turned to Puppet Labs, which recently picked up an additional $30 million in financing from VMware as part of an ambitious effort to create software-defined data centers that allow organizations to manage IT resources using a simple declarative language versus mastering an actual programming language.

“There’s some of overlap between what Puppet does and VMware currently provides,” said Ronnie Colville, an analyst with the IT Operations Group at Gartner. “But there’s also a lot of capability in terms of what can be done with Puppet that VMware today simply can’t provide.”

Unfortunately, change of this magnitude doesn’t come easy to most IT organizations. In fact, when it comes to IT automation, Coville suggests IT organizations are “bipolar” in terms of either fully embracing it or completely resisting it.

To overcome that resistance, Puppet Lab chief executive Luke Kaines believes that IT organizations should start small. “What we really recommend is that IT organizations put a small team of people together that are really committed to making a change,” he said. “It’s really about managing IT at a new level of scale. We very rarely see organizations trying to eliminate IT jobs because of automation.”

Nevertheless, while IT automation makes a lot of sense on paper, there are no end of cultural and technical issues that have conspired to limit adoption. Many IT professionals not only worry that IT automation will eliminate their job; they contend that existing products don’t provide enough transparency into the process or simply don’t work as well as the scripts they have developed themselves. On top of that, many IT organizations don’t have very mature IT management processes in place, which makes it difficult to automate a process that doesn’t formally exist.

Adam Jacob, chief customer officer for Opscode and creator of Chef, says one big reason a lot of organizations implode when it comes to IT automation is that they fail to take a holistic approach to the problem. Individual teams within the IT organization wind up automating specific layers of computing such as the network or storage layer. The final result is a lot of incompatible points of automation.

“The worst thing you can do is try to automate IT without really knowing how the underlying processes really work,” Jacob said. “You want to run IT like a factory, but it’s not like factories just spring up overnight. IT automation is a journey. You have to be prepared for the fact that you’re at some point going to fall down multiple times during the course of that journey.”

Fortunately, the fundamental need to address everything from managing virtualization to the DevOps crisis is forcing more organizations to finally confront their IT management issues.

Those IT management issues go way beyond infrastructure, suggested Josh Greenbaum, principal of Enterprise Applications Consulting: “In a lot of instances IT organizations are required under the terms of their licensing agreement to use application-specific tools from SAP or Oracle.”

While Puppet Labs and Opscode are garnering a lot of attention these days with in the open-source community, it’s the applications issue that’s starting to attract new open-source competitors.

Cloud Sidekick, for example, recently emerged from stealth mode to deliver commercial support around Cato, an open source IT automation platform based on technology developed by T-Mobile. According to Cloud Sidekick chief execurtive Dennis Ehle, the company’s Maestro IT automation management platform was developed to make it easier to manage environments where a lot of complex inter-dependencies exist between application and systems.

“The goal is identify all the logical dependencies no matter how dynamic they are,” Ehle said. “In some instances a virtual machine might only be up for a day. We want to make it possible for organizations to self-service their IT needs even in the most complex heterogeneous environments.”

Richard Merwin, manager of ERP research and development at North Carolina State University, said that capability is what attracted NCSU to the platform. While NCSU uses Puppet to manage some of its x86 server environments, when it came to its PeopleSoft applications from Oracle, it was looking for automation framework that could handle a lot more complexity.

“We’re an open-source first shop,” he said. “Maestro and Cato allow us to do more in the way of customization, run queries and load balancing. But it’s still a work in progress.”

Of course, providers of commercial IT automation software that include IBM, CA Technologies, UC4 Software, BMC Software and Advanced Systems Concepts are not sitting idly by. While they all acknowledge that open source has played a role in making IT automation software more accessible, the increasing complexity of the IT environment will require approaches to IT automation that more easily scale not only across departments but also multiple organizations.

In the meantime, as a general trend, IT automation is becoming more closely aligned with the whole “DevOps” movement. Without some ability to automate, IT organizations simply won’t be able to keep pace with the volume of code generated by developers who have embraced agile methodologies. But whatever the cause, the one thing for certain is that the way data centers have been managed for the last 20 years (or more) will not scale into the future.

“No matter how you want look at it,” said Puppet Labs’ Kanies. “IT organizations are not going to be able to keep pace relying on their existing tools and skills.”

Image: bannosuke/Shutterstock.com

Helping to protect IT environments from cyber attacks and comply with tightening compliance standards, SIEM systems are becoming the cornerstone for security paradigms implemented by a growing number of organizations.

In a previous post, we explained what a SIEM system actually is — why organizations require it to start with, the components it is comprised of, and how it helps mitigate attacks. One of the conclusions reached in that article was that SIEM is not actually a single tool in itself, but is instead comprised of multiple monitoring and analysis components.

There are proprietary platforms that do offer an all-in-one SIEM solution, such as LogRhythm, QRadar, and ArcSight. These solutions can become rather expensive, especially in the long run and in larger organizations, and so more and more companies are on the search for an open source SIEM platform.

But is there an open source platform that includes all the basic SIEM ingredients?

The simple answer is — no. There is no all-in-one perfect open source SIEM system. Existing solutions either lack core SIEM capabilities, such as event correlation and reporting or require combining with other tools. As always, though, there are some good contenders, and in this article, we take a look at six of these platforms. Wifi wpa psk password hacker.

We will follow up this article with a similar analysis of proprietary tools.

1. OSSIM

The open source version of AlienVault’s Unified Security Management (USM) offering, OSSIM is probably one of the more popular open source SIEM platforms. OSSIM includes key SIEM components, namely event collection, processing and normalization, and most importantly — event correlation.

OSSIM combines native log storage and correlation capabilities with numerous open source projects in order to build a complete SIEM. The list of open source projects included in OSSIM includes: FProbe, Munin, Nagios, NFSen/NFDump, OpenVAS, OSSEC, PRADS, Snort, Suricata and TCPTrack.

The inclusion of OpenVAS is of particular interest, as OpenVAS is used both for vulnerability assessment by correlating IDS logs with vulnerability scanner results.

As one would expect, the open source OSSIM is not as feature rich as its commercial “older brother”. Both solutions work fine for small deployments, but OSSIM users experience significant performance issues at scale, ultimately driving them towards the commercial offering. Log management capabilities in the open source version of OSSIM, for example, are virtually non-existent.

2. The ELK Stack

The ELK stack, or the Elastic Stack as it is being renamed these days, is arguably the most popular open-source tool used today as a building block in a SIEM system. A building block — yes. A complete SIEM system — no, since there is plenty of room for debate about whether or not the ELK Stack qualifies as an “all in one” SIEM system.

The ELK stack consists of the open-source products Elasticsearch, Logstash, Kibana and the Beats family of log shippers.

Logstash is a log aggregator that can collect and process data from almost any data source. It can filter, process, correlate and generally enhance any log data that it collects. Elasticsearch is the storage engine and one of the best solutions in its field for storing and indexing time-series data. Kibana is the visualization layer in the stack and an extremely powerful one at that. Beats include a variety of light-weight log shippers that are responsible for collecting the data and shipping it into the stack via Logstash.

Logstash uses a wide array of input plugins to collect logs. However, it can also accept input from more purpose-built solutions like OSSEC or Snort (see below). Combined, the ELK Stack’s log processing, storage and visualization capabilities are functionally unmatched. For the purposes of SIEM however, the ELK Stack — at least in its raw open source format, is missing some key components.

First and foremost, there is no built-in reporting or alerting capability. This is a known pain point not only for users trying to use the stack for security but also for more common use cases — IT operations for example. Alerting can be added by using the X-Pack, a commercial product by Elastic, or by adding an adding open source security add-ons.

There are also no built-in security rules that can be used. This makes the stack a bit more costly to handle, both in terms of resources and operational costs.

3. OSSEC

Open Source Office Automation

OSSEC is a popular open source Host Intrusion Detection System (HIDS) that works with various operating systems, including Linux, Windows, MacOS, Solaris, as well as OpenBSD and FreeBSD.

OSSEC itself is broken into two main components: the manager (or server), responsible for collecting the log data from the different data sources, and the agents — applications that are responsible for collecting and processing the logs and making them easier to analyze.

The OSSEC project itself does not include a visualization layer. There was a UI which was deprecated, and instead, the recommendation is to use external visualization tools such as Kibana and Grafana.

OSSEC directly monitors a number of parameters on a host. This includes log files, file integrity, rootkit detection, and Windows registry monitoring. OSSEC can perform log analysis from other network services, including most of the popular open source FTP, mail, DNS, database, web, firewall, and network-based IDS solutions. OSSEC can also analyze logs from a number of commercial network services and security solutions.

OSSEC has a number of alerting options and can be used as part of automated intrusion detection or active response solutions. OSSEC has a primitive log storage engine. By default, log messages from host agents are not retained. Once analyzed, OSSEC deletes these logs unless the <logall> option is included in the OSSEC manager’s ossec.conf file. If this option is enabled, OSSEC stores the incoming logs from agents in a text file that is rotated daily.

Whether or not OSSEC can be counted as an “all in one” SIEM system is debatable. OSSEC definitely does the hard work involved in implementing a SIEM system: it collects data and analyzes it, but lacks some of the core log management and analysis components required. It’s worth pointing out that the OSSEC project has been forked by other HIDS solutions (e.g. Wazuh) that extend OSSEC functionality and make it a more complete SIEM option.

4. Apache Metron

Evolving from Cisco’s OpenSOC platform and first released in 2016, Apache Metron is a relatively new player in the industry and another example of a security framework that combines multiple open source projects into one platform.

From an architectural perspective, Metron relies on other Apache projects for collecting, streaming and processing security data. Apache Nifi and Metron probes collect data from security data sources which is then pushed into separate Apache Kafka topics. Events are subsequently parsed and normalized into standard JSON and then enriched and in some cases labeled. Alerts can be triggered if certain event types are identified. For visualization, Kibana is used (albeit an outdated version)

For storage, events are indexed and persisted in Apache Hadoop and either Elasticsearch or Solr based on the organization’s preferences. On top of this data, Metron provides an interface for centralizing the analysis of the data with alert summaries and enriched data.

One of Metron’s strongest features is it’s pluggable and extensible architecture. Bro, pycapa and fastcapa sensors for example, can be used to ship specific data into Metron. Using Stellar, a simple DSL, users can write their own functions for transforming collected data. An extensive REST API allows users to interact with Metron, so users can for example programmatically manage alerts.

Being relatively young, Metron still lacks in some aspects. Metron can only be installed on a limited number of operating systems and environments though it does support automation scenarios with Ansible and installation via Docker (Mac and Windows only). The UI is a bit immature and does not support authentication for example.

5. SIEMonster

SIEMonster is another young SIEM player but an extremely popular one as well, with over 100,000 downloads in just two years. SIEMonster is based on open source technology and is available for free and as a paid solution (Premium and MSSP multi-tenancy).

While SIEMonster uses its own “monster” terminology to name the different SIEM functions within the system (e.g. Kraken), the underlying components are well known open source technologies. The ELK Stack is used for collection (Filebeat and Logstash), processing, storage and visualization of the security data collected. RabbitMQ is used for queuing. SearchGuard is used for encryption and authentication on top of Elasticsearch and ElastAlert for alerting. A fork of OSSEC Wazuh for HIDS. The list goes on.

From a functionality perspective, SIEMonster includes all the goodies an analyst could wish for, each accessed via a main menu — the Kibana UI for searching and visualizing data, a MineMeld UI for threat intelligence, Alerts for creating and managing event-based notifications. Additional integrated open source tools are DRADIS, OpenAudit, and FIR.

SIEMonster can be deployed on the cloud using Docker containers, meaning easier portability across systems, but also on VMs and bare metal (Mac, Ubuntu, CentOS, and Debian). Documentation is extensive, though an online version is missing.

6. Prelude

Similar to OSSIM, Prelude is a SIEM framework that unifies various other open source tools. And like OSSIM, it is also an open source version of thecommercial tool by the same name. Prelude aims to fill the roles that tools like OSSEC and Snort leave out.

Prelude accepts logs and events from multiple sources and stores them all in a single location using theIntrusion Detection Message Exchange Format (IDMEF). It provides filtering, correlation, alerting, analysis, and visualization capabilities.

Open Source Automation Tools For Siebel Logos

Again, like OSSIM, the open source version of Prelude is significantly limited when compared to the commercial offering in all of these capabilities which is probably why it is not very popular. Quoting the official documentation: “Prelude OSS is aimed for evaluation, research and test purpose on very small environments. Please note that Prelude OSS performances are way lower than the Prelude SIEM edition.”

No “one ring to rule them all”

A complete SIEM solution includes the ability to collect information from various data sources, retain that information for an extended period of time, correlate between different events, create correlation rules or alerts, analyze the data and monitor it with visualizations and dashboards.

Open Source Automation Server

Answering a lot of these requirements, it is no coincidence that the ELK Stack is used by many of the open source SIEM systems listed in this article. OSSEC Wazuh, SIEMonster, Metron — all have ELK beneath the hood. But taken on its own, ELK lacks some key SIEM components, such as correlation rules and incident management.

Based on the analysis above, the simple conclusion is that there are no clear winners to the title “an all-in-one open source SIEM solution”. When implementing a SIEM system based on the solutions above, you will most likely find yourself limited as far as functionality is concerned or combining with additional open source tools.

Open source tools used for SIEM are versatile and powerful. But, they require a great deal of expertise, and above all — time to deploy properly. It is for this reason that commercial offerings still dominate the SIEM landscape, even when open-source tools lie at the core of those commercial offerings.

Having 80% of your SIEM solution handled for you is better than having to do it all by yourself. Commercial solutions handle installation, basic configuration, and provide filters, correlation configurations, and visualization designs for the most common use cases. Don’t underestimate the value of these commercial features: there are a seemingly unlimited number of things to monitor in today’s datacenters, and none of us have time to manually configure applications to watch them all.