Publications
Characterizing phishing threats with natural language processing
September 28, 2015
Conference Paper
Published in:
2015 IEEE Conf. on Communications and Network Security (CNS), 28-30 September 2015.
Topic:
R&D area:
R&D group:
Summary
Spear phishing is a widespread concern in the modern network security landscape, but there are few metrics that measure the extent to which reconnaissance is performed on phishing targets. Spear phishing emails closely match the expectations of the recipient, based on details of their experiences and interests, making them a popular propagation vector for harmful malware. In this work we use Natural Language Processing techniques to investigate a specific real-world phishing campaign and quantify attributes that indicate a targeted spear phishing attack. Our phishing campaign data sample comprises 596 emails - all containing a web bug and a Curriculum Vitae (CV) PDF attachment - sent to our institution by a foreign IP space. The campaign was found to exclusively target specific demographics within our institution. Performing a semantic similarity analysis between the senders' CV attachments and the recipients' LinkedIn profiles, we conclude with high statistical certainty (p < 10^-4) that the attachments contain targeted rather than randomly selected material. Latent Semantic Analysis further demonstrates that individuals who were a primary focus of the campaign received CVs that are highly topically clustered. These findings differentiate this campaign from one that leverages random spam.
Summary
Spear phishing is a widespread concern in the modern network security landscape, but there are few metrics that measure the extent to which reconnaissance is performed on phishing targets. Spear phishing emails closely match the expectations of the recipient, based on details of their experiences and interests, making them a...
READ MORE
Very large graphs for information extraction (VLG) - detection and inference in the presence of uncertainty
September 21, 2015
Project Report
Published in:
MIT Lincoln Laboratory Report VLG-2
Summary
In numerous application domains relevant to the Department of Defense and the Intelligence Community, data of interest take the form of entities and the relationships between them, and these data are commonly represented as graphs. Under the Very Large Graphs for Information Extraction effort--a one year proof-of-concept study--MIT LL developed novel techniques for anomalous subgraph detection, building on tools in the signal processing research literature. This report documents the technical results of this effort. Two datasets--a snapshot of Thompson Reuters' Web of Science database and a stream of web proxy logs--were parsed, and graphs were constructed from the raw data. From the phenomena in these datasets, several algorithms were developed to model the dynamic graph behavior, including a preferential attachment mechanism with memory, a streaming filter to model a graph as a weighted average of its past connections, and a generalized linear model for graphs where connection probabilities are determined by additional side information or metadata. A set of metrics was also constructed to facilitate comparison of techniques. The study culminated in a demonstration of the algorithms on the datasets of interest, in addition to simulated data. Performance in terms of detection, estimation, and computational burden was measured according to the metrics. Among the highlights of this demonstration were the detection of emerging coauthor clusters in the Web of Science data, detection of botnet activity in the web proxy data after 15 minutes (which took 10 days to detect using state-of-the-practice techniques), and demonstration of the core algorithm on a simulated 1-billion-vertex graph using a commodity computing cluster.
Summary
In numerous application domains relevant to the Department of Defense and the Intelligence Community, data of interest take the form of entities and the relationships between them, and these data are commonly represented as graphs. Under the Very Large Graphs for Information Extraction effort--a one year proof-of-concept study--MIT LL developed...
READ MORE
Cyber network mission dependencies
September 18, 2015
Technical Report
Published in:
MIT Lincoln Laboratory Report TR-1189
Topic:
R&D area:
R&D group:
Summary
Cyber assets are critical to mission success in every arena of the Department of Defense. Because all DoD missions depend on cyber infrastructure, failure to secure network assets and assure the capabilities they enable will pose a fundamental risk to any defense mission. The impact of a cyber attack is not well understood by warfighters or leadership. It is critical that the DoD develop better cognizance of Cyber Network Mission Dependencies (CNMD). This report identifies the major drivers for mapping missions to network assets, introduces existing technologies in the mission-mapping landscape, and proposes directions for future development.
Summary
Cyber assets are critical to mission success in every arena of the Department of Defense. Because all DoD missions depend on cyber infrastructure, failure to secure network assets and assure the capabilities they enable will pose a fundamental risk to any defense mission. The impact of a cyber attack is...
READ MORE
The AFRL-MITLL WMT15 System: there's more than one way to decode it!
September 17, 2015
Conference Paper
Published in:
Proc. 10th Workshop on Statistical Machine Translation, 17-18 September 2015, pp. 112-9.
Topic:
R&D area:
Summary
This paper describes the AFRL-MITLL statistical MT systems and the improvements that were developed during the WMT15 evaluation campaign. As part of these efforts we experimented with a number of extensions to the standard phrase-based model that improve performance on the Russian to English translation task creating three submission systems with different decoding strategies. Out of vocabulary words were addressed with named entity postprocessing.
Summary
This paper describes the AFRL-MITLL statistical MT systems and the improvements that were developed during the WMT15 evaluation campaign. As part of these efforts we experimented with a number of extensions to the standard phrase-based model that improve performance on the Russian to English translation task creating three submission systems...
READ MORE
Evaluation of the baseline NEXRAD icing hazard project
September 17, 2015
Conference Paper
Published in:
37th Conference on Radar Meteorology, 14-18 September 2015
Summary
MIT Lincoln Laboratory has developed an icing hazard product that is now operational throughout the NEXRAD network. This initial version of the Icing Hazard Levels (IHL) algorithm is predicated on the presence of graupel as determined by the NEXRAD Hydrometeor Classification Algorithm (HCA). Graupel indicates that rime accretion on ice crystal aggregates is present. It is inferred that the riming process occurs at the altitude that HCA reports graupel as well as to some vertical depth above. To capture some of that depth, temperature and relative humidity interest fields are computed from meteorological model data based on the technique used in the National Center for Atmospheric Research's Current Icing Potential Product and utilized within IHL as warranted. A critical aspect of the IHL development has focused on the verification of the presence of icing. Two methods are used. For the first, pilot reports of icing (PIREPs) are used to score the performance of IHL. Since PIREPs are provided with inherent time and space uncertainties, a buffer of influence is associated with each PIREP when scoring IHL. Results show the IHL as configured is an effective indicator of a potential icing hazard when HCA graupel classifications are present. Results also show the importance of radar volume coverage pattern selection in detecting weak returns in winter weather. For the second, in situ icing missions were performed within range of a dual pol NEXRAD to provide quantitative data to identify the presence of supercooled liquid water. Comparisons of in situ data to HCA classifications show that HCA graupel indications do not fully expose the icing hazard and these findings are being used to direct future attention of IHL development. This paper will describe the verification method and performance assessment of the IHL initial capability.
Summary
MIT Lincoln Laboratory has developed an icing hazard product that is now operational throughout the NEXRAD network. This initial version of the Icing Hazard Levels (IHL) algorithm is predicated on the presence of graupel as determined by the NEXRAD Hydrometeor Classification Algorithm (HCA). Graupel indicates that rime accretion on ice...
READ MORE
Sampling large graphs for anticipatory analytics
September 15, 2015
Conference Paper
Published in:
HPEC 2015: IEEE Conf. on High Performance Extreme Computing, 15-17 September 2015.
Topic:
R&D area:
R&D group:
Summary
The characteristics of Big Data - often dubbed the 3V's for volume, velocity, and variety - will continue to outpace the ability of computational systems to process, store, and transmit meaningful results. Traditional techniques for dealing with large datasets often include the purchase of larger systems, greater human-in-the-loop involvement, or more complex algorithms. We are investigating the use of sampling to mitigate these challenges, specifically sampling large graphs. Often, large datasets can be represented as graphs where data entries may be edges, and vertices may be attributes of the data. In particular, we present the results of sampling for the task of link prediction. Link prediction is a process to estimate the probability of a new edge forming between two vertices of a graph, and it has numerous application areas in understanding social or biological networks. In this paper we propose a series of techniques for the sampling of large datasets. In order to quantify the effect of these techniques, we present the quality of link prediction tasks on sampled graphs, and the time saved in calculating link prediction statistics on these sampled graphs.
Summary
The characteristics of Big Data - often dubbed the 3V's for volume, velocity, and variety - will continue to outpace the ability of computational systems to process, store, and transmit meaningful results. Traditional techniques for dealing with large datasets often include the purchase of larger systems, greater human-in-the-loop involvement, or...
READ MORE
Secure architecture for embedded systems
September 15, 2015
Conference Paper
Published in:
HPEC 2015: IEEE Conf. on High Performance Extreme Computing, 15-17 September 2015.
Topic:
Summary
Devices connected to the internet are increasingly the targets of deliberate and sophisticated attacks. Embedded system engineers tend to focus on well-defined functional capabilities rather than "obscure" security and resilience. However, "after-the-fact" system hardening could be prohibitively expensive or even impossible. The co-design of security and resilience with functionality has to overcome a major challenge; rarely can the security and resilience requirements be accurately identified when the design begins. This paper describes an embedded system architecture that decouples secure and functional design aspects.
Summary
Devices connected to the internet are increasingly the targets of deliberate and sophisticated attacks. Embedded system engineers tend to focus on well-defined functional capabilities rather than "obscure" security and resilience. However, "after-the-fact" system hardening could be prohibitively expensive or even impossible. The co-design of security and resilience with functionality has...
READ MORE
Improving big data visual analytics with interactive virtual reality
September 15, 2015
Conference Paper
Published in:
HPEC 2015: IEEE Conf. on High Performance Extreme Computing, 15-17 September 2015.
Topic:
R&D area:
R&D group:
Summary
For decades, the growth and volume of digital data collection has made it challenging to digest large volumes of information and extract underlying structure. Coined 'Big Data', massive amounts of information has quite often been gathered inconsistently (e.g from many sources, of various forms, at different rates, etc.). These factors impede the practices of not only processing data, but also analyzing and displaying it in an efficient manner to the user. Many efforts have been completed in the data mining and visual analytics community to create effective ways to further improve analysis and achieve the knowledge desired for better understanding. Our approach for improved big data visual analytics is two-fold, focusing on both visualization and interaction. Given geo-tagged information, we are exploring the benefits of visualizing datasets in the original geospatial domain by utilizing a virtual reality platform. After running proven analytics on the data, we intend to represent the information in a more realistic 3D setting, where analysts can achieve an enhanced situational awareness and rely on familiar perceptions to draw in-depth conclusions on the dataset. In addition, developing a human-computer interface that responds to natural user actions and inputs creates a more intuitive environment. Tasks can be performed to manipulate the dataset and allow users to dive deeper upon request, adhering to desired demands and intentions. Due to the volume and popularity of social media, we developed a 3D tool visualizing Twitter on MIT's campus for analysis. Utilizing emerging technologies of today to create a fully immersive tool that promotes visualization and interaction can help ease the process of understanding and representing big data.
Summary
For decades, the growth and volume of digital data collection has made it challenging to digest large volumes of information and extract underlying structure. Coined 'Big Data', massive amounts of information has quite often been gathered inconsistently (e.g from many sources, of various forms, at different rates, etc.). These factors...
READ MORE
Enabling on-demand database computing with MIT SuperCloud database management system
September 15, 2015
Conference Paper
Published in:
HPEC 2015: IEEE Conf. on High Performance Extreme Computing, 15-17 September 2015.
Topic:
R&D area:
R&D group:
Summary
The MIT SuperCloud database management system allows for rapid creation and flexible execution of a variety of the latest scientific databases, including Apache Accumulo and SciDB. It is designed to permit these databases to run on a High Performance Computing Cluster (HPCC) platform as seamlessly as any other HPCC job. It ensures the seamless migration of the databases to the resources assigned by the HPCC scheduler and centralized storage of the database files when not running. It also permits snapshotting of databases to allow researchers to experiment and push the limits of the technology without concerns for data or productivity loss if the database becomes unstable.
Summary
The MIT SuperCloud database management system allows for rapid creation and flexible execution of a variety of the latest scientific databases, including Apache Accumulo and SciDB. It is designed to permit these databases to run on a High Performance Computing Cluster (HPCC) platform as seamlessly as any other HPCC job...
READ MORE
Big data strategies for data center infrastructure management using a 3D gaming platform
September 15, 2015
Conference Paper
Published in:
HPEC 2015: IEEE Conf. on High Performance Extreme Computing, 15-17 September 2015.
Topic:
R&D area:
R&D group:
Summary
High Performance Computing (HPC) is intrinsically linked to effective Data Center Infrastructure Management (DCIM). Cloud services and HPC have become key components in Department of Defense and corporate Information Technology competitive strategies in the global and commercial spaces. As a result, the reliance on consistent, reliable Data Center space is more critical than ever. The costs and complexity of providing quality DCIM are constantly being tested and evaluated by the United States Government and companies such as Google, Microsoft and Facebook. This paper will demonstrate a system where Big Data strategies and 3D gaming technology is leveraged to successfully monitor and analyze multiple HPC systems and a lights-out modular HP EcoPOD 240a Data Center on a singular platform. Big Data technology and a 3D gaming platform enables the relative real time monitoring of 5000 environmental sensors, more than 3500 IT data points and display visual analytics of the overall operating condition of the Data Center from a command center over 100 miles away. In addition, the Big Data model allows for in depth analysis of historical trends and conditions to optimize operations achieving even greater efficiencies and reliability.
Summary
High Performance Computing (HPC) is intrinsically linked to effective Data Center Infrastructure Management (DCIM). Cloud services and HPC have become key components in Department of Defense and corporate Information Technology competitive strategies in the global and commercial spaces. As a result, the reliance on consistent, reliable Data Center space is...
READ MORE