潜在特征关系大小单双倍投公式(LFRM)是图结构化数据的生成大小单双倍投公式,用于学习图中每个节点的二元矢量表示。二进制向量表示节点在一个或多个社区中的成员资格。 LFRM miller2009非参数的核心是一个重叠的随机块大小单双倍投公式,它将任何一对节点之间的链路概率定义为其社区成员矢量的双线性函数。此外,使用非参数贝叶斯先验(印度自助餐流程)可以从数据中自动学习社区数量。然而,尽管其具有吸引力的特性,LFRM中的推断仍然是一个挑战,并且通常通过MCMC方法进行。这可能很慢并且可能需要很长时间才能收敛。在这项工作中,我们开发了一个基于小方差渐近的框架,用于参数贝叶斯LFRM。这导致了一个目标函数,它保留了LFRM的非参数贝叶斯风格,同时使我们能够为这个大小单双倍投公式设计易于实现的(使用通用或专门的优化例程)并且在实践中快速实现。我们在几个基准数据集上得到了结果证明我们的算法与MCMC等方法具有竞争力,同时速度更快。
translated by 谷歌翻译
图是对关系数据建模的基本抽象。然而,图形本质上是离散的和组合的,并且适用于机器学习任务的学习表示引起统计和计算挑战。在这项工作中,我们提出了Graphite一个算法框架,用于使用深度变量生成大小单双倍投公式对图中节点的表示进行非监督学习。我们的大小单双倍投公式基于变化的编码器(VAE),并使用图形神经网络来参数化生成大小单双倍投公式(即解码器)和推理大小单双倍投公式(即编码器)。由于直接在生成大小单双倍投公式中的图的空间局部结构,图形神经网络的使用直接结合了归纳偏差。我们通过kernelembeddings绘制了我们框架的连接和近似推理。根据经验,Graphite优于竞争方法,用于密度估计,链接预测以及基于合成和基准数据集的节点分类。
translated by 谷歌翻译
Networks observed in real world like social networks, collaboration networksetc., exhibit temporal dynamics, i.e. nodes and edges appear and/or disappearover time. In this paper, we propose a generative, latent space based,statistical model for such networks (called dynamic networks). We consider thecase where the number of nodes is fixed, but the presence of edges can varyover time. Our model allows the number of communities in the network to bedifferent at different time steps. We use a neural network based methodology toperform approximate inference in the proposed model and its simplified version.Experiments done on synthetic and real world networks for the task of communitydetection and link prediction demonstrate the utility and effectiveness of ourmodel as compared to other similar existing approaches.
translated by 谷歌翻译
Graph embedding is an effective method to represent graph data in a low dimensional space for graph analytics. Most existing embedding algorithms typically focus on preserving the topological structure or minimizing the reconstruction errors of graph data, but they have mostly ignored the data distribution of the latent codes from the graphs, which often results in inferior embedding in real-world graph data. In this paper, we propose a novel adversarial graph embedding framework for graph data. The framework encodes the topological structure and node content in a graph to a compact representation , on which a decoder is trained to reconstruct the graph structure. Furthermore, the latent representation is enforced to match a prior distribution via an adversarial training scheme. To learn a robust embedding, two variants of adversarial approaches , adversarially regularized graph autoen-coder (ARGA) and adversarially regularized vari-ational graph autoencoder (ARVGA), are developed. Experimental studies on real-world graphs validate our design and demonstrate that our algorithms outperform baselines by a wide margin in link prediction, graph clustering, and graph visual-ization tasks.
translated by 谷歌翻译
Beta-Bernoulli processes, also known as Indian buffet processes, are nonparametric priors that allow generative models to automatically infer the number of features in datasets. However, inference for these models proves to be challenging, often relying on specific forms of the likelihood for computational tractability. We propose to amortize inference using a variational autoencoder trained via gradient descent, allowing for arbitrary likelihood models. Our model extends previously considered mean field variational methods with a structured posterior and new developments in the training of discrete variable VAEs. We experimentally demonstrate a Beta-Bernoulli process VAE that learns decomposable latent features and allows for scalable inference of arbitrary likelihoods on large datasets.
translated by 谷歌翻译
图形嵌入旨在将图形转换为向量,以便于后续图形分析任务,如链接预测和图形聚类。图形嵌入的大多数方法着重于保留图形结构或最小化图形数据的重建错误。他们大多忽略了潜码的嵌入分布,不幸的是,在许多情况下,这可能导致较差的表现。在本文中,我们提出了用于图嵌入的anovel对称正则化框架。通过使用图卷积网络作为编码器,我们的框架将拓扑信息和节点内容嵌入到矢量表示中,从中进一步构建图解码器以重建输入图。应用对抗训练原则来强制执行我们的潜在代码以匹配先前的高斯或统一分布。在此框架的基础上,我们推导出对抗大小单双倍投公式的两个变量,对偶正则化图自动编码器(ARGA)及其变分版本,对称正则化变分图形编码器(ARVGA),以有效地学习图形嵌入。我们还利用ARGA和ARVGA的其他潜在变体来更深入地了解我们的设计。 12个链接预测算法和20个图谱聚类算法的实验结果比较验证了我们的解决方案。
translated by 谷歌翻译
我们研究了与图表表示学习相关的两个基本任务:链接预测和半监督节点分类。我们提出了一种新的自动编码器体系结构,能够学习本地图形结构和可用节点特征的联合表示,用于链路预测和节点分类的多任务学习。我们的自动编码体系结构在单个学习阶段进行端到端的高效训练,同时执行链路预测和节点分类,而以前的相关方法需要多个难以优化的训练步骤。我们对九个基准图形结构数据集上的大小单双倍投公式进行了全面的实证评估,并展示了图形表示学习的重要改进相关方法。参考代码和数据可在https://github.com/vuptran/graph-representation-learning上获得
translated by 谷歌翻译
The recently developed variational autoencoders (VAEs) have proved to be an effective confluence of the rich repre-sentational power of neural networks with Bayesian methods. However, most work on VAEs use a rather simple prior over the latent variables such as standard normal distribution , thereby restricting its applications to relatively simple phenomena. In this work, we propose hierarchical non-parametric variational autoencoders, which combines tree-structured Bayesian nonparametric priors with VAEs, to enable infinite flexibility of the latent representation space. Both the neural parameters and Bayesian priors are learned jointly using tailored variational inference. The resulting model induces a hierarchical structure of latent semantic concepts underlying the data corpus, and infers accurate representations of data instances. We apply our model in video representation learning. Our method is able to discover highly interpretable activity hierarchies, and obtain improved clustering accuracy and generalization capacity based on the learned rich representations.
translated by 谷歌翻译
关于潜变量大小单双倍投公式和深度学习的互补强度的结合,最近有很多令人兴奋的工作。潜变量建模使得通过条件独立属性明确指定大小单双倍投公式约束变得容易,而深度学习使得利用强大的函数逼近器参数化条件似然成为可能。虽然这些“深潜变量”大小单双倍投公式为建模现实世界现象提供了丰富,灵活的框架,但存在一些困难:条件可能性的深度参数化通常会使后向推理难以处理,而且变量目标通常通过引入非可微性点来使反向传播复杂化。本教程通过变分推理的镜头深入探讨了这些问题。
translated by 谷歌翻译
Latent variable models for network data extract a summary of the relational structure underlying an observed network. The simplest possible models subdivide nodes of the network into clusters; the probability of a link between any two nodes then depends only on their cluster assignment. Currently available models can be classified by whether clusters are disjoint or are allowed to overlap. These models can explain a "flat" clustering structure. Hierarchical Bayesian models provide a natural approach to capture more complex dependencies. We propose a model in which objects are characterised by a latent feature vector. Each feature is itself partitioned into disjoint groups (subclusters), corresponding to a second layer of hierarchy. In experimental comparisons, the model achieves significantly improved predictive performance on social and biological link prediction tasks. The results indicate that models with a single layer hierarchy oversimplify real networks.
translated by 谷歌翻译
许多现代无监督或半监督机器学习算法依赖于贝叶斯概率大小单双倍投公式。这些大小单双倍投公式通常难以处理,因此需要进行近似推断。变分推理(VI)通过解决优化问题,使我们可以通过更简单的变分分布来近似高维贝叶斯后验。这种方法已成功应用于各种大小单双倍投公式和大规模应用。在这篇综述中,我们对变分推断的最新趋势进行了概述。我们首先介绍standardmean字段变分推理,然后回顾以下方面的最新进展:(a)可扩展的VI,包括随机近似,(b)通用VI,它将VI的适用性扩展到一大类其他难以处理的大小单双倍投公式,如非共轭大小单双倍投公式,(c)准确的VI,其中包括超出平均场近似或非典型差异的变分大小单双倍投公式,以及(d)摊销的VI,它利用推理网络实现推理超局部潜在变量。最后,我们提供了有希望的未来研究方向的摘要。
translated by 谷歌翻译
建模和生成图表是研究生物学,工程和社会科学网络的基础。然而,由于图形的非唯一,高维性质以及agiven图中边缘之间存在的复杂的非局部依赖性,对图形上的复杂分布进行建模然后从这些分布进行有效采样是具有挑战性的。在这里,我们提出GraphRNN,一个深度自回归大小单双倍投公式,它解决了上述挑战,并且近似于任何关于其结构的最小假设的图的分布。 GraphRNN学习通过对一组有代表性的图形进行训练来生成图形,并将图形生成过程分解为一系列节点和边缘形式,以到目前为止生成的图形结构为条件。为了定量评估GraphRNN的性能,我们引入了基于最大平均差异的基准数据集,基线和新评估指标套件,它们测量图形集之间的距离。 Ourexperiments显示GraphRNN明显优于所有基线,学习生成与目标集的结构特征匹配的不同图形,同时还缩放到比以前的深度大小单双倍投公式大50倍的图形。
translated by 谷歌翻译
图表上的机器学习是一项重要且无处不在的任务,其应用范围从药物设计到社交网络中的友情推荐。该领域的主要挑战是找到一种表示或编码图形结构的方法,以便机器学习大小单双倍投公式可以轻松利用它。传统上,机器学习方法依赖于用户定义的启发式方法来提取编码关于图形的结构信息的特征(例如,度统计或核函数)。然而,近年来,使用基于深度学习和非线性降维的技术,自动学习将图形结构编码为低维嵌入的方法出现了激增。在这里,我们提供了图表表示学习领域关键进展的概念性回顾,包括基于矩阵分解的方法,基于随机游走的算法和图神经网络。我们回顾了嵌入单个节点的方法以及嵌入整个(子)图的方法。在这样做的过程中,我们开发了一个统一的框架来描述这些最近的方法,并且我们强调了许多重要的应用和未来工作的方向。
translated by 谷歌翻译
神经变分推断的最新进展催生了文艺复兴时期的隐变量大小单双倍投公式。在本文中,我们介绍了用于文本的生成和条件大小单双倍投公式的通用变分推理框架。虽然传统的变分方法导出了对潜在变量的可分配分布的解析近似,但在这里我们构造了一个以离散文本输入为条件的推理网络,以提供变分分布。我们在两个非常不同的文本大小单双倍投公式应用,生成文档建模和监督问题回答中验证了这个框架。我们的神经变分文档大小单双倍投公式将连续随机文档表示与词袋生成大小单双倍投公式相结合,并在两个标准测试语料库中实现最低报告的困惑。神经答案选择大小单双倍投公式在注意机制内采用随机表示层来提取问题和答案对之间的语义。在两个问题基准上,这个大小单双倍投公式超过了之前公布的所有基准。
translated by 谷歌翻译
We develop a scalable algorithm for posterior inference of overlapping communities in large networks. Our algorithm is based on stochastic variational inference in the mixed-membership stochastic blockmodel (MMSB). It naturally interleaves subsampling the network with estimating its community structure. We apply our algorithm on ten large, real-world networks with up to 60,000 nodes. It converges several orders of magnitude faster than the state-of-the-art algorithm for MMSB, finds hundreds of communities in large real-world networks, and detects the true communities in 280 benchmark networks with equal or better accuracy compared to other scalable algorithms.
translated by 谷歌翻译
大小单双倍投公式 Graphs, such as social networks, word co-occurrence networks, and communication networks, occur naturally in various real-world applications. Analyzing them yields insight into the structure of society, language, and different patterns of communication. Many approaches have been proposed to perform the analysis. Recently, methods which use the representation of graph nodes in vector space have gained traction from the research community. In this survey, we provide a comprehensive and structured analysis of various graph embedding techniques proposed in the literature. We first introduce the embedding task and its challenges such as scalability, choice of dimensionality, and features to be preserved, and their possible solutions. We then present three categories of approaches based on factorization methods, random walks, and deep learning, with examples of representative algorithms in each category and analysis of their performance on various tasks. We evaluate these state-of-the-art methods on a few common datasets and compare their performance against one another. Our analysis concludes by suggesting some potential applications and future directions. We finally present the open-source Python library we developed, named GEM (Graph Embedding Methods, available at https://github.com/palash1992/GEM), which provides all presented algorithms within a unified interface to foster and facilitate research on the topic.
translated by 谷歌翻译
从联合分布的样本中学习忠实的有向无环图(DAG)是一个具有挑战性的组合问题,因为图形节点数量的难以处理的搜索空间超指数。最近的一次突破将问题表述为具有确保非离子性的结构约束的连续优化(Zheng等,2018)。作者应用了线性结构方程大小单双倍投公式(SEM)和最小二乘损失函数的方法,这些方法在统计上是合理的,但仍然是有限的。由于能够捕获复杂非线性映射的深度学习的广泛成功,在这项工作中,我们提出了陡峭的生成大小单双倍投公式,并应用结构约束的变体来学习DAG。生成大小单双倍投公式的核心是一个变分自动编码器,由一个新的图形神经网络结构参数化,我们用它来标记DAG-GNN。除了更丰富的容量之外,所提出的大小单双倍投公式的一个优点是它自然地处理离散变量以及向量值变量。我们证明了在合成数据集上,所提出的方法为非线性生成的样本学习更准确的图形;并且在具有离散变量的基准数据集上,学习的图形合理地接近全局估计。该代码位于\ url {https://github.com/fishmoon1234/DAG-GNN}。
translated by 谷歌翻译
Training of discrete latent variable models remains challenging because passing gradient information through discrete units is difficult. We propose a new class of smoothing transformations based on a mixture of two overlapping distributions , and show that the proposed transformation can be used for training binary latent models with either directed or undirected priors. We derive a new variational bound to efficiently train with Boltzmann machine priors. Using this bound, we develop DVAE++, a generative model with a global discrete prior and a hierarchy of convo-lutional continuous variables. Experiments on several benchmarks show that overlapping transformations outperform other recent continuous relaxations of discrete latent variables including Gumbel-Softmax (Maddison et al., 2016; Jang et al., 2016), and discrete variational autoencoders (Rolfe, 2016).
translated by 谷歌翻译
变分自动编码器(VAE)是表达性潜变量大小单双倍投公式,可用于从训练数据中学习复杂的概率分布。然而,所得大小单双倍投公式的质量至关重要地依赖于推理大小单双倍投公式的表现。我们介绍了Adversarial VariationalBayes(AVB),这是一种使用任意表达式推理大小单双倍投公式训练变分自动编码器的技术。我们通过引入辅助判别网络来实现这一目标,该网络允许重新解释双人游戏的最大似然问题,从而在VAE和生成对抗网络(GAN)之间建立原则连接。我们证明了在非参数极限中,我们的方法为生成大小单双倍投公式的参数提供了精确的最大似然赋值,以及给出观察的潜在变量的精确后验分布。与将VAE与GAN相结合的竞争方法相反,我们的方法具有明确的理论上的理由,保留了标准变分自动编码器的大部分优点并且易于实现。
translated by 谷歌翻译
The likelihood model of high dimensional data X n can often be expressed as p(X n |Z n , θ), where θ := (θ k) k∈[K] is a collection of hidden features shared across objects, indexed by n, and Z n is a non-negative factor loading vector with K entries where Z nk indicates the strength of θ k used to express X n. In this paper, we introduce random function priors for Z n for modeling correlations among its K dimensions Z n1 through Z nK , which we call population random measure embedding (PRME). Our model can be viewed as a generalized paintbox model (Broderick et al., 2013) using random functions, and can be learned efficiently with neural networks via amortized variational inference. We derive our Bayesian nonparametric method by applying a representation theorem on separately exchangeable discrete random measures.
translated by 谷歌翻译