Publications
Characterization of disinformation networks using graph embeddings and opinion mining
November 26, 2019
Conference Paper
Published in:
2019 European Intelligence and Security Informatics Conference, EISIC, 26-27 November 2019.
Topic:
Summary
Global social media networks' omnipresent access, real time responsiveness and ability to connect with and influence people have been responsible for these networks' sweeping growth. However, as an unintended consequence, these defining characteristics helped create a powerful new technology for spread of propaganda and false information. We present a novel approach for characterizing disinformation networks on social media and distinguishing between different network roles using graph embeddings and hierarchical clustering. In addition, using topic filtering, we correlate the node characterization results with proxy opinion estimates.We plan to study opinion dynamics using signal processing on graphs approaches using longer-timescale social media datasets with the goal to model and infer influence among users in social media networks.
Summary
Global social media networks' omnipresent access, real time responsiveness and ability to connect with and influence people have been responsible for these networks' sweeping growth. However, as an unintended consequence, these defining characteristics helped create a powerful new technology for spread of propaganda and false information. We present a novel...
READ MORE
Investigation of the relationship of vocal, eye-tracking, and fMRI ROI time-series measures with preclinical mild traumatic brain injury
October 19, 2019
Abstract
Published in:
Neuroscience 2019
Summary
In this work, we are examining correlations between vocal articulatory features, ocular smooth pursuit measures, and features from the fMRI BOLD response in regions of interest (ROI) time series in a high school athlete population susceptible to repeated head impact within a sports season. Initial results have indicated relationships between vocal features and brain ROIs that may show which components of the neural speech networks effected are effected by preclinical mild traumatic brain injury (mTBI). The data used for this study was collected by Purdue University on 32 high school athletes over the entirety of a sports season (Helfer, et al., 2014), and includes fMRI measurements made pre-season, in-season, and postseason. The athletes are 25 male football players and 7 female soccer players. The Immediate Post-Concussion Assessment and Cognitive Testing suite (ImPACT) was used as a means of assessing cognitive performance (Broglio, Ferrara, Macciocchi, Baumgartner, & Elliott, 2007). The test is made up of six sections, which measure verbal memory, visual memory, visual motor speed, reaction time, impulse control, and a total symptom composite. Using each test, a threshold is set for a change in cognitive performance. The threshold for each test is defined as a decline from baseline that exceeds one standard deviation, where the standard deviation is computed over the change from baseline across all subjects’ test scores. Speech features were extracted from audio recordings of the Grandfather Passage, which provides a standardized and phonetically balanced sample of speech. Oculomotor testing included two experimental conditions. In the smooth pursuit condition, a single target moving circularly, at constant speed. In the saccade condition, a target was jumped between one of three location along the horizontal midline of the screen. In both trial types, subjects visually tracked the targets during the trials, which lasted for one minute. The fMRI features are derived from the bold time-series data from resting state fMRI scans of the subjects. The pre-processing of the resting state fMRI and accompanying structural MRI data (for Atlas registration) was performed with the toolkit CONN (Whitfield-Gabrieli & Nieto-Castanon, 2012). Functional connectivity was generated using cortical and sub-cortical atlas registrations. This investigation will explores correlations between these three modalities and a cognitive performance assessment.
Summary
In this work, we are examining correlations between vocal articulatory features, ocular smooth pursuit measures, and features from the fMRI BOLD response in regions of interest (ROI) time series in a high school athlete population susceptible to repeated head impact within a sports season. Initial results have indicated relationships between...
READ MORE
Multi-Objective Graph Matching via Signal Filtering
October 11, 2019
Journal Article
Published in:
IEEE Signal Processing Magazine Special Issue on GSP [submitted]
Topic:
R&D area:
Summary
In this white paper we propose a new method which exploits tools from graph signal processing to solve the graph matching problem, the problem of estimating the correspondence between the vertex sets of two graphs. We recast the graph matching problem as matching multiple similarity matrices where the similarities are computed between filtered signals unique to eachnode. Using appropriate graph filters, these similarity matrices can emphasize long or short range behavior and the method will implicitly search for similarities between the graphs and at multiple scales. Our method shows substantial improvementsover standard methods which use the raw adjacency matrices, especially in low-information environments.
Summary
In this white paper we propose a new method which exploits tools from graph signal processing to solve the graph matching problem, the problem of estimating the correspondence between the vertex sets of two graphs. We recast the graph matching problem as matching multiple similarity matrices where the similarities are...
READ MORE
An Eye on the Storm: Tracking Power Outages via the Internet of Things
October 4, 2019
Conference Paper
Published in:
Grace Hopper Celebration 2019 [submitted]
Topic:
R&D area:
Summary
Assessing the extent of power outages in the wake of disasters is a crucial but daunting challenge. We developed a prototype to estimate and map the severity and location of power outages throughout an event by taking advantage of IoT as a non-traditional power-sensing network. We present results used by FEMA and other responders during multiple major hurricanes, such as Harvey, Irma, and Maria.
Summary
Assessing the extent of power outages in the wake of disasters is a crucial but daunting challenge. We developed a prototype to estimate and map the severity and location of power outages throughout an event by taking advantage of IoT as a non-traditional power-sensing network. We present results used by...
READ MORE
Uncovering human trafficking networks through text analysis
October 1, 2019
Conference Paper
Published in:
2019 Grace Hopper Celebration, 1-4 October 2019.
Topic:
R&D area:
Summary
Human trafficking is a form of modern-day slavery affecting an estimated 40 million victims worldwide, primarily through the commercial sexual exploitation of women and children. In the last decade, the advertising of victims has moved from the streets to websites on the Internet, providing greater efficiency and anonymity for sex traffickers. This shift has allowed traffickers to list their victims in multiple geographic areas simultaneously, while also improving operational security by using multiple methods of electronic communication with buyers; complicating the ability of law enforcement to disrupt these illicit organizations. In this presentation, we present a novel unsupervised template matching algorithm for analyzing and detecting complex organizations operating on adult service websites. We apply this method to a large corpus of adult service advertisements retrieved from backpage.com, and show that the networks identified through the algorithm match well with surrogate truth data derived from phone number networks in the same corpus. Further exploration of the results show that the proposed method provides deeper insights into the complex structures of sex trafficking organizations, not possible through networks derived from phone numbers alone. This method provides a powerful new capability for law enforcement to more completely identify and gather evidence about trafficking operations
Summary
Human trafficking is a form of modern-day slavery affecting an estimated 40 million victims worldwide, primarily through the commercial sexual exploitation of women and children. In the last decade, the advertising of victims has moved from the streets to websites on the Internet, providing greater efficiency and anonymity for sex...
READ MORE
Corpora design and score calibration for text dependent pronunciation proficiency recognition
September 20, 2019
Conference Paper
Published in:
8th ISCA Workshop on Speech and Language Technology in Education, SLaTe 2019, 20-21 September 2019.
Topic:
R&D area:
Summary
This work investigates methods for improving a pronunciation proficiency recognition system, both in terms of phonetic level posterior probability calibration, and in ordinal utterance level classification, for Modern Standard Arabic (MSA), Spanish and Russian. To support this work, utterance level labels were obtained by crowd-sourcing the annotation of language learners' recordings. Phonetic posterior probability estimates extracted using automatic speech recognition systems trained in each language were estimated using a beta calibration approach [1] and language proficiency level was estimated using an ordinal regression [2]. Fusion with language recognition (LR) scores from an i-vector system [3] trained on 23 languages is also explored. Initial results were promising for all three languages and it was demonstrated that the calibrated posteriors were effective for predicting pronunciation proficiency. Significant relative gains of 16% mean absolute error for the ordinal regression and 17% normalized cross entropy for the binary beta regression were achieved on MSA through fusion with LR scores.
Summary
This work investigates methods for improving a pronunciation proficiency recognition system, both in terms of phonetic level posterior probability calibration, and in ordinal utterance level classification, for Modern Standard Arabic (MSA), Spanish and Russian. To support this work, utterance level labels were obtained by crowd-sourcing the annotation of language learners'...
READ MORE
Using K-means in SVR-based text difficulty estimation
September 20, 2019
Conference Paper
Published in:
8th ISCA Workshop on Speech and Language Technology in Education, SLaTE, 20-21 September 2019.
Topic:
R&D area:
Summary
A challenge for second language learners, educators, and test creators is the identification of authentic materials at the right level of difficulty. In this work, we present an approach to automatically measure text difficulty, integrated into Auto-ILR, a web-based system that helps find text material at the right level for learners in 18 languages. The Auto-ILR subscription service scans web feeds, extracts article content, evaluates the difficulty, and notifies users of documents that match their skill level. Difficulty is measured on the standard ILR scale with language-specific support vector machine regression (SVR) models built from vectors incorporating length features, term frequencies, relative entropy, and K-means clustering.
Summary
A challenge for second language learners, educators, and test creators is the identification of authentic materials at the right level of difficulty. In this work, we present an approach to automatically measure text difficulty, integrated into Auto-ILR, a web-based system that helps find text material at the right level for...
READ MORE
State-of-the-art speaker recognition for telephone and video speech: the JHU-MIT submission for NIST SRE18
September 15, 2019
Conference Paper
Published in:
Interspeech, 15-19 September 2019.
Topic:
R&D area:
Summary
We present a condensed description of the joint effort of JHUCLSP, JHU-HLTCOE, MIT-LL., MIT CSAIL and LSE-EPITA for NIST SRE18. All the developed systems consisted of xvector/i-vector embeddings with some flavor of PLDA backend. Very deep x-vector architectures–Extended and Factorized TDNN, and ResNets– clearly outperformed shallower xvectors and i-vectors. The systems were tailored to the video (VAST) or to the telephone (CMN2) condition. The VAST data was challenging, yielding 4 times worse performance than other video based datasets like Speakers in the Wild. We were able to calibrate the VAST data with very few development trials by using careful adaptation and score normalization methods. The VAST primary fusion yielded EER=10.18% and Cprimary= 0.431. By improving calibration in post-eval, we reached Cprimary=0.369. In CMN2, we used unsupervised SPLDA adaptation based on agglomerative clustering and score normalization to correct the domain shift between English and Tunisian Arabic models. The CMN2 primary fusion yielded EER=4.5% and Cprimary=0.313. Extended TDNN x-vector was the best single system obtaining EER=11.1% and Cprimary=0.452 in VAST; and 4.95% and 0.354 in CMN2.
Summary
We present a condensed description of the joint effort of JHUCLSP, JHU-HLTCOE, MIT-LL., MIT CSAIL and LSE-EPITA for NIST SRE18. All the developed systems consisted of xvector/i-vector embeddings with some flavor of PLDA backend. Very deep x-vector architectures–Extended and Factorized TDNN, and ResNets– clearly outperformed shallower xvectors and i-vectors. The...
READ MORE
Improving robustness to attacks against vertex classification
August 5, 2019
Conference Paper
Published in:
15th Intl. Workshop on Mining and Learning with Graphs, 5 August 2019.
Topic:
R&D area:
Summary
Vertex classification—the problem of identifying the class labels of nodes in a graph—has applicability in a wide variety of domains. Examples include classifying subject areas of papers in citation networks or roles of machines in a computer network. Recent work has demonstrated that vertex classification using graph convolutional networks is susceptible to targeted poisoning attacks, in which both graph structure and node attributes can be changed in an attempt to misclassify a target node. This vulnerability decreases users' confidence in the learning method and can prevent adoption in high-stakes contexts. This paper presents work in progress aiming to make vertex classification robust to these types of attacks. We investigate two aspects of this problem: (1) the classification model and (2) the method for selecting training data. Our alternative classifier is a support vector machine (with a radial basis function kernel), which is applied to an augmented node feature-vector obtained by appending the node’s attributes to a Euclidean vector representing the node based on the graph structure. Our alternative methods of selecting training data are (1) to select the highest-degree nodes in each class and (2) to iteratively select the node with the most neighbors minimally connected to the training set. In the datasets on which the original attack was demonstrated, we show that changing the training set can make the network much harder to attack. To maintain a given probability of attack success, the adversary must use far more perturbations; often a factor of 2–4 over the random training baseline. Even in cases where success is relatively easy for the attacker, we show that the classification and training alternatives allow classification performance to degrade much more gradually, with weaker incorrect predictions for the attacked nodes.
Summary
Vertex classification—the problem of identifying the class labels of nodes in a graph—has applicability in a wide variety of domains. Examples include classifying subject areas of papers in citation networks or roles of machines in a computer network. Recent work has demonstrated that vertex classification using graph convolutional networks is...
READ MORE
Discriminative PLDA for speaker verification with X-vectors
May 12, 2019
Conference Paper
Published in:
International Conference on Acoustics, Speech, and Signal Processing, May 2019 [submitted]
Topic:
R&D area:
Summary
This paper proposes a novel approach to discriminative training ofprobabilistic linear discriminant analysis (PLDA) for speaker veri-fication with x-vectors. The Newton Method is used to discrimi-natively train the PLDA model by minimizing the log loss of ver-ification trials. By diagonalizing the across-class and within-classcovariance matrices as a pre-processing step, the PLDA model canbe trained without relying on approximations, and while maintain-ing important properties of the underlying covariance matrices. Thetraining procedure is extended to allow for efficient domain adapta-tion. When applied to the Speakers in the Wild and SRE16 tasks, theproposed approach provides significant performance improvementsrelative to conventional PLDA.
Summary
This paper proposes a novel approach to discriminative training ofprobabilistic linear discriminant analysis (PLDA) for speaker veri-fication with x-vectors. The Newton Method is used to discrimi-natively train the PLDA model by minimizing the log loss of ver-ification trials. By diagonalizing the across-class and within-classcovariance matrices as a pre-processing step, the...
READ MORE