Notes | YuyaoGe's Website

Softmax Regression and Its Optimization

Thu, 21 Mar 2024 00:00:00 +0000

Introduction

A refresher on the fundamentals

This article is part of my lecture notes from the Deep Learning Systems course taught by Tianqi Chen and J. Zico Kolter at CMU.

It covers the second lecture of the course: 2 - ML Refresher / Softmax Regression.

Defining the Data

In this section we define the input data used throughout this article.

Consider a $k$-class classification problem, in which we are given:

Training data: $x^{(i)}\in \mathbb{R}^n$, $y^{(i)}\in {1,…,k}$ $for$ $i = 1,..,m$.
n is the dimensionality of the input data
k is the number of classes/labels
m is the number of training examples

That completes the definition of all input data used in this article.

Linear Mapping

In this section we define the hypothesis function $h$ that maps the input data $x$ to $k$ classes.

We will use a linear function for this purpose, give its formal statement, and prove that the function is indeed linear.

Vector Form

Since we want to classify data $x$ of dimension $n$ into $k$ classes, we need to state this task formally.

For a $k$-class classification problem, we want to map an $n$-dimensional input vector $x\in \mathbb{R}^n$ into a $k$-dimensional solution space, that is $$ h{:}\mathbb{R}^n\to\mathbb{R}^k $$ where $h_i(x)$ denotes the confidence that the input $x$ belongs to class $i$.

This process can be expressed with a linear mapping function: $$ {h_\theta(x)=\theta^Tx} $$ where $\theta \in \mathbb{R}^{n\times k}$.

This completes the formal statement of the mapping function.

Why is $ h_\theta(x) = \theta^T x $ linear?

A linear function must satisfy additivity and homogeneity.

Additivity: Suppose we have two $n$-dimensional input vectors $x_1$ and $ x_2$. Then for the linear hypothesis function $h$ we have: $$ h_\theta(x_1 + x_2) = \theta^T (x_1 + x_2) $$ Since matrix multiplication is compatible with vector addition, by the distributive law: $$h_\theta(x_1 + x_2) = \theta^T x_1 + \theta^T x_2 $$ and the right-hand side equals transforming each vector separately and then summing: $$h_\theta(x_1 + x_2) = h_\theta(x_1) + h_\theta(x_2) $$ so additivity holds.

Homogeneity: If we take a scalar multiple $cx$ of $ x$, where $c$ is an arbitrary real number, the output of the linear hypothesis function $h$ is: $$ h_\theta(cx) = \theta^T (cx) $$ By the distributive law: $$ h_\theta(cx) = c(\theta^T x) $$ This shows that a scalar multiple transformed by ( h ) is exactly the transformation scaled by that scalar: $$ h_\theta(cx) = c h_\theta(x) $$

so homogeneity holds.

Matrix multiplication therefore satisfies both additivity and homogeneity, which proves that $h_\theta(x)$ is a linear function.

Matrix Form

In the previous subsection, $x$ was a vector of shape $1 \times n$. In this subsection we stack $m$ such vectors $x$ into a matrix of shape $m \times n$:

$$ X\in\mathbb{R}^{m\times n}=\begin{bmatrix}-{x^{(1)}}^T-\\\vdots\\-{x^{(m)}}^T-\end{bmatrix} $$

For a single vector $x$, its label $y$ is a real number. Likewise, when $m$ vectors $x$ are stacked into a matrix, the labels $y$ form a matrix of shape $m \times 1$:

$$ y\in\{1,...,k\}^m=\begin{bmatrix}y^{(1)}\\\vdots\\y^{(m)}\end{bmatrix} $$

Loss Functions

In this section we discuss the definitions, merits, and drawbacks of two loss functions — the 0-1 loss and the cross-entropy loss.

A loss function maps the model’s parameter matrix to a real number, and that number measures how good the hypothesis function $h_\theta(x)$ is.

0-1 Loss

For a classification problem, how do we judge whether a hypothesis function $h_\theta(x)$ is good?

A very intuitive idea is to assign $1$ when the prediction is wrong and $0$ when it is right. This is called the “0-1 loss function”. Its value depends on whether the classifier $h$’s prediction $h_\theta(x) $ on input $x $ matches the true label $y$. If the prediction is correct, the loss is 0; if it is wrong, the loss is 1. In short, the function asks only whether the classifier made a mistake, penalizing errors and nothing else. Formally:

$$ \ell_{err}(h(x),y)=\begin{cases}0&\text{if }\operatorname{argmax}_ih_i(x)=y\\1&\text{otherwise}\end{cases} $$

The advantage of this loss is its simplicity. Unfortunately, it is of little help when optimizing the model parameters, because it cannot be used with gradient-based methods — it has no gradient at all! When the prediction becomes correct, the loss jumps abruptly to 0, so there is no meaningful gradient direction to guide the parameter update.

Here, $ \text{argmax}_i h_i(x) $ denotes the class with the highest score among all the scores the classifier outputs.

Softmax and Cross-entropy Loss

The idea behind the “0-1 loss” of the previous subsection can be summed up as “all or nothing”: 1 for correct, 0 for wrong. Yet even among wrong answers, there is a difference between being slightly off and being wildly off.

With that in mind, we would like to turn the output of the hypothesis function $h_\theta(x)$ from a rigid all-or-nothing verdict into a “probability”. Here we can use the softmax function, which converts any vector of real numbers into a probability distribution. For class $i $, the probability predicted by the model, $p(\text{label} = i) $, is computed as the exponential of that class’s score $ h_i(x) $ divided by the sum of the exponentials of all class scores, which ensures that the predicted probabilities sum to 1. Formally:

$$ z_i=p(\text{label}=i)=\frac{\exp\bigl(h_i(x)\bigr)}{\sum_{j=1}^k\exp\bigl(h_j(x)\bigr)}\Longleftrightarrow z\equiv\text{softmax}\bigl(h(x)\bigr) $$

Dividing the output by the sum of all outputs is easy to understand: normalizing this way makes every class output fall between 0 and 1 and makes the outputs sum to 1. The less obvious part is why we exponentiate $ h_i(x) $ in the first place.

There are two main reasons:

Exponentiating turns the hypothesis function’s outputs into positive values. This is necessary for a valid probability distribution, since probabilities cannot be negative.
The exponential curve is increasing, and — most importantly — its slope grows steadily, meaning that a small change along the x-axis can produce a large change along the y-axis. This helps sharply distinguish the classes with higher raw logits when computing probabilities.

With the softmax function in hand, we can compute the loss with the cross-entropy loss, stated formally as:

$$ \ell_{ce}(h(x),y)=-\log p(\text{label}=y)=-h_y(x)+\log\sum_{j=1}^k\exp\left(h_j(x)\right) $$

Note that the cross-entropy loss is essentially the $-log$ of the softmax output for class $y$.

Adding the $log$ serves two purposes:

It maps values in the interval (0, 1) to the interval from negative infinity to 0, so that small changes in probability are amplified in the loss value, providing better discrimination.
When the predicted probability is close to that of the true label, the gradient of the loss becomes small and the model learns more slowly. Conversely, when the prediction differs greatly from the true label, the gradient becomes markedly larger and the model learns faster. This lets the model learn more effectively, adjusting course quickly when it frequently makes wrong predictions.

Adding the $-$ turns negative values into positive ones, so that a larger discrepancy means a larger loss, and vice versa.

Optimization

As noted above, the loss function lets us judge how good the weight parameters of the hypothesis function $h_\theta(x)$ are.

So if the weights in $h_\theta(x)$ are not good enough, how should we optimize them?

Definition

Broadly speaking, we want to optimize the weight parameters $\theta$ so as to minimize the output of the loss function. Formally: $$ \underset{\theta}{\operatorname*{minimize}}\frac1m\underset{i=1}{\operatorname*{\sum}}\ell(h_\theta(x^{(i)}),y^{(i)}) $$ where $i$ is the index of the current sample, $h$ is the hypothesis function, and $y$ is the sample’s true label (ground truth).

Substituting the cross-entropy loss $\ell_{ce}$ for $\ell$ gives: $$ \underset{\theta}{\operatorname*{minimize}}\frac1m\sum_{i=1}^m\ell_{ce}(\theta^Tx^{(i)},y^{(i)}) $$ That gives us the definition of the optimization problem.

Concretely, then, how do we carry out the optimization?

Gradient-based Optimization

Gradients

For a function $f{:\mathbb{R}^{n\times k}\to\mathbb{R}}$ that takes a matrix as input and returns a real number, its gradient is defined as the matrix of partial derivatives, stated formally as:

$$ \nabla_\theta f(\theta)\in\mathbb{R}^{n\times k}=\begin{bmatrix}\dfrac{\partial f(\theta)}{\partial\theta_{11}}&...&\dfrac{\partial f(\theta)}{\partial\theta_{1k}}\\\vdots&\ddots&\vdots\\\dfrac{\partial f(\theta)}{\partial\theta_{n1}}&...&\dfrac{\partial f(\theta)}{\partial\theta_{nk}}\end{bmatrix} $$

Combining this with the discussion above, $f$ here stands for the loss function.

Mathematically, the gradient points in the direction of steepest local ascent of the loss function.

In optimization, therefore, we adjust the parameters along the negative gradient in order to minimize the loss.

Learning Rate

Armed with the gradient, a powerful tool, we can now update the weight parameters.

Note, however, that we update the weights round by round. We therefore need a scaling factor $\alpha$ that defines how much the weights are updated in each round of training. This scaling factor $\alpha$ is called the learning rate. When updating weights with the gradient, each step can be stated formally as: $$ \begin{aligned}\theta:=\theta-\alpha\nabla_\theta f(\theta)\end{aligned} $$ where:

$\theta$ is the weight parameter to be optimized.
$\alpha$ is the learning rate, a positive number controlling the size of each update step.
$\nabla_\theta f(\theta)$ is the gradient of the loss function $f(\theta) $ with respect to the parameter $ \theta$.

This method is the well-known Gradient Descent. It works by computing the gradient of the loss function at the current parameters, then updating the parameters along the negative gradient direction, iterating until convergence.

In gradient descent, choosing an appropriate learning rate is crucial to optimization performance. The three subplots below show the paths taken by gradient descent in a two-dimensional parameter space under different learning rates $\alpha$, where the contour lines represent the value of the loss function. From left to right, the learning rates are 0.05, 0.2, and 0.42. The goal of gradient descent is to find the global minimum of the loss function, which is the innermost contour ring in the figure.

From these three subplots we can observe that:

When the learning rate is small (0.05, left), the descent path is long and many more iterations are needed to reach the minimum. This means we must run many rounds of iteration before finding the optimal weights.
When the learning rate is moderate (0.2, middle), gradient descent approaches the minimum fairly quickly.
When the learning rate is large (0.42, right), the updates may overshoot, causing the descent path to wander around the minimum and struggle to settle down.

We therefore need to choose the learning rate carefully. The reasons are:

Convergence speed: if the learning rate is too small, gradient descent may converge to the minimum very slowly, increasing the number of iterations needed to reach the optimum and making training take much longer.
Oscillation: if the learning rate is too large, the parameter updates may overshoot, causing the algorithm to jump around the minimum or even past it, so that the loss actually increases. This makes the algorithm diverge and never find the minimum.
Avoiding poor local minima: a well-chosen learning rate can help the algorithm converge accurately to the global minimum of the loss function, or to a good local minimum, rather than to a mediocre local minimum or a saddle point.
Stability: an overly large learning rate can cause oscillation around the minimum instead of convergence, whereas an appropriate learning rate ensures that the parameter updates approach the optimum smoothly and stably.

Stochastic Gradient Descent

The previous subsection introduced gradient descent, giving us a paradigm for optimizing weight parameters. In practice, however, we more often update weights using an optimization method called Stochastic Gradient Descent (SGD).

In machine learning, when the loss function is a sum of individual losses, we would rather not compute the gradient over all samples just to perform a single parameter update, because that is expensive in both computation and time.

Instead, we can take only a small portion of the dataset (called a minibatch) at a time to compute the gradient and update the parameters. This uses fewer computational resources and lets us make many more parameter updates in the same amount of time.

We define a minibatch formally as: $$ X\in\mathbb{R}^{B\times n},y\in{1,…,k}^B $$

The stochastic gradient descent update can then be expressed formally as: $$ \theta:=\theta-\frac{\alpha}{B}\sum_{i=1}^{B}\nabla_{\theta}\ell(h_{\theta}(x^{(i)}),y^{(i)}) $$

Computing the Gradient of the Cross-entropy Loss

In the previous section we learned how to use gradients to update the weight parameters. We are now just one step away from our goal — computing the gradient.

So how do we compute the gradient of the cross-entropy loss? $$ \nabla_\theta\ell_{ce}(\theta^Tx,y)=? $$ By the chain rule, the expression above can be written as: $$ \frac\partial{\partial\theta}\ell_{ce}(\theta^Tx,y) = \frac{\partial\ell_{ce}(\theta^Tx,y)}{\partial\theta^Tx}\frac{\partial\theta^Tx}{\partial\theta} $$ The right-hand side splits into two terms, namely: $$ \frac{\partial\ell_{ce}(\theta^Tx,y)}{\partial\theta^Tx} \tag{1} $$ and $$ \frac{\partial\theta^Tx}{\partial\theta} \tag{2} $$ For ${(1)}$:

Remember the formula for the cross-entropy loss? Let us take the partial derivative of the loss with respect to the output $h_i$ of some class.

$$ \begin{aligned} \begin{aligned}\frac{\partial\ell_{ce}(h,y)}{\partial h_i}\end{aligned}& \begin{aligned}=\frac{\partial}{\partial h_i}\left(-h_y+\log\sum_{j=1}^k\exp h_j\right)\end{aligned} \\ &=-1\{i=y\}+\frac{\exp h_i}{\sum_{j=1}^k\exp h_j} \end{aligned} $$

Here we make a surprising discovery: the derivative of the cross-entropy loss contains $\frac{\exp h_i}{\sum_{j=1}^k\exp h_j}$! Looks familiar, doesn’t it? That is exactly the expansion of Softmax(h(x))!

We can therefore simplify the result above to $\begin{aligned}\nabla_h\ell_{ce}(h,y)=z-e_y\end{aligned}$, where $z=\mathrm{softmax}(h)$ and $e_y$ is the one-hot vector corresponding to the correct class.

For ${(2)}$:

The derivative is easy to obtain: $$ \frac{\partial\theta^Tx}{\partial\theta} = x $$ So equation $(0)$ can be expressed as:

$$ \begin{aligned} \begin{aligned}\frac{\partial}{\partial\theta}\ell_{ce}(\theta^Tx,y)\end{aligned}& \begin{aligned}=\frac{\partial\ell_{ce}(\theta^Tx,y)}{\partial\theta^Tx}\frac{\partial\theta^Tx}{\partial\theta}\end{aligned} \\ &=({z-e_y})(x),\quad\text{where }z=\text{softmax}(\theta^Tx) \end{aligned} $$

Just one more step to go!

Let us do a little more work to align the matrix shapes…

So, when the input is a vector $x$, the gradient of the cross-entropy loss with respect to the weight matrix $\theta$ is: $$ \nabla_\theta\ell_{ce}(\theta^Tx,y)\in\mathbb{R}^{n\times k}=x(z-e_y)^T $$ Extending the input to the matrix $X$ formed by stacking all vectors $x$ in a batch, the expression becomes: $$ \nabla_\theta \ell_{ce}(X\theta, y) \in \mathbb{R}^{n \times k} = X^T(Z - I_y) $$ where:

$X$ is the feature matrix of the entire data batch.
$Z$ is the matrix of predicted probabilities produced by the hypothesis function for the whole batch. Each row is the predicted probability distribution of one sample, obtained by passing $X\theta$ through the softmax function.
$ I_y $ is a matrix in which each row is the one-hot vector of the correct class for one sample.

With that, we have completed the gradient computation for the cross-entropy loss, and we can use the gradient to update the weight matrix!

Conclusion

In this article we covered the softmax function, the cross-entropy loss, gradients, gradient descent, stochastic gradient descent, and how to use gradient descent to optimize weight parameters.

Stay tuned for the follow-up articles!

Graph Data Mining Notes — NIPS 2020 Workshop

Thu, 14 Mar 2024 00:00:00 +0000

Preface

These are my notes from the workshop given by Google’s Graph Mining Team at NIPS 2020.

The project homepage is Graph Mining & Learning, and the PDF version is available at Mining and Learning with Graphs at Scale.

Introduction

What Is a Graph?

A graph is a form of data that represents relationships between entities.

Here, an entity corresponds to a node, and a relationship corresponds to an edge.

In general, graphs have the following characteristics:

A large number of edges
Multiple types of edges and multiple types of nodes
Highly complex structure

Graphs can therefore be used to represent social networks, transportation networks, and much more.

Types of Graphs

Broadly speaking, graphs fall into two categories: natural graphs and similarity graphs.

Natural Graphs

In a natural graph, the edges come from external, real-world circumstances or data. In other words, the entities and relationships the graph represents genuinely exist in the real world. For example, every transaction in a payment network can be seen as an edge linking the payer and the payee; in a social network, a friendship between two people can likewise be expressed as an edge; in a road network, roads connect different locations; and in co-click or co-watch data, if two items are clicked or watched by the same person, an edge can be drawn between them. What defines this kind of graph is that the existence and nature of its edges are determined by real-world relationships or interactions.

Similarity Graphs

In contrast to natural graphs, the edges of a similarity graph come from the similarity between nodes. Here we start with a large collection of metadata or data and then build the graph structure by measuring the similarity or distance between data points. For instance, given a set of articles, we can connect them based on the similarity of their content (say, keyword overlap or topical similarity) to form a graph. In this case, edges are created by analyzing and computing over the properties of the data itself, rather than being derived from relationships that already exist externally.

Note: when working with similarity graphs, the original metadata of each node remains available to us. For high-dimensional metadata, preprocessing and precomputation can avoid a great deal of redundant computation and therefore save considerable time.

Why Use Graphs?

Computation on Abstract Concepts

“Computation on abstract concepts” means handling complex relationships with graphs. Interpersonal ties in a social network, route connections in a transportation system, and causal relationships in scientific research are all typically complicated, yet much of this data can be reduced to interactions and connections among entities — that is, to a graph. Graphs are a powerful tool for understanding and manipulating such relationships. They help us not only understand and analyze local information (between directly connected entities) but also abstract over that information, extracting general patterns or principles from specific instances and local situations. Abstracting local information this way lets us pull useful global information out of the data as a whole: identifying key nodes across an entire network (opinion leaders in a social network, say), discovering group structure through community detection, or understanding how the overall structure shapes individual behavior and system performance.

Computation on Multimodal Data

In many scenarios we have to handle data of several modalities at once — visual, textual, semantic information — and these data are interrelated. Graphs let us relate data across modalities and compute over them jointly.

The Global View

Graph structure can help us discover patterns, groups, or relationships in data, and can even quantify concepts that are intrinsically hard to measure directly.

Take the figure above as an example: given an ambiguous picture of an apple, we can compute its similarity to a cluster of Apple Inc. images and to a cluster of natural apple images, and thereby determine which cluster the picture belongs to.

The Local View

By computing over graph structure, a graph can supply information that goes beyond the attributes of any single node. Concretely, a node on its own tells us only so much; but by computing over the nodes and edges around it, we can learn a great deal more.

Take the figure above as an example: looking at a single pixel, we have no idea what it means. But by looking at the pixels around it, we can tell that the pixel comes from a cat’s eye.

The Graph Mining Toolbox

This part of the talk introduces the toolbox developed and used by Google’s Graph Mining Team. Since it comes up again later, we skip the details here.

Application Cases

Spam, Fraud, and Abuse Detection

For detecting spam, fraud, and abuse — particularly in trust and safety work — the Google team presents two core ideas and methods: anomaly detection via density clustering, and label propagation.

Anomaly detection via density clustering:

Statistically improbable dense clusters correlate strongly with malicious behavior. In other words, if we find unusually dense clusters of nodes in a graph representation of the data, those clusters very likely correspond to spammers, fraud, or other kinds of abuse. Graph mining tools can identify these unusual dense clusters and thus detect and prevent malicious behavior. This is essential for keeping products such as YouTube and the ad platforms safe and trustworthy.
Label propagation:

Starting from known bad actors, we use the graph structure to identify neighboring nodes that are likely problematic as well. The intuition is that malicious users or entities tend to sit close to one another in the graph or to be connected in some way — for example, they may jointly participate in fraud or spam campaigns. By propagating a label (such as “malicious”) from known bad actors to the nodes they connect to, we can surface additional suspicious actors. This effectively broadens the detection net and uncovers entities that were never directly flagged as malicious.

Improving Machine Learning Models

Relationship discovery:

The way social networks find “people you may know” is by analyzing the social graph. Graph information can reveal non-obvious connections: even when a link is not apparent from direct social interaction, graph analysis can expose latent ties such as mutual friends, shared interests, or similar social paths.
Feature extraction:

Signals produced from graphs — clusters, personalized PageRank (PPR) vectors, and graph embeddings — are highly useful for training downstream machine learning models. These graph-derived features can substantially improve model performance by providing deep information about relationships between entities. In multimodal models, graph data can serve as yet another modality, joining text, images, and audio to form a larger whole. This lets machine learning models draw on a richer dataset and thereby achieve a more comprehensive understanding and more accurate predictions.
A case study:

Consider the “visually similar images” feature of Google Images. By analyzing similarity between images, one can build a graph in which nodes are images and edges represent visual similarity. Even when a user specifies no explicit search criteria, the system can find other images that are visually similar to a given one.

Efficient Computation

Resource efficiency: communication overhead

In distributed computing systems, large datasets must be partitioned and spread across compute nodes, and the resulting communication overhead can be reduced with graph partitioning algorithms. Applying graph partitioning in Google Driving Directions, for instance, optimizes backend processing and makes road network data processing far more efficient.
Data efficiency and active learning

Graphs let us answer queries such as “which points in my dataset are the most diverse,” which can drive an active learning loop. Active learning is a strategy that allows a model to request annotations for specific data points so as to learn and improve as efficiently as possible; it is especially suited to settings where labeled data is scarce. Graph-based semi-supervised learning suits models with little data: it can leverage a small amount of labeled data together with a large amount of unlabeled data, using graph structure to improve learning.

Application Stories

Modeling COVID Spread with Spatio-Temporal Graph Neural Networks

Fundamentals

A deep machine learning model can be described as learning a function $f(x)$, where $x$ is a carefully chosen set of features. The model’s intermediate states — the embeddings — capture the complex interactions among features in a high-dimensional space. The model learns the intricate patterns and relationships in the data by adjusting these embeddings so as to minimize a loss function.

Deep learning models can automatically learn and abstract the complex relationships between input data and output targets, even when those relationships are not obvious or direct at the outset. Deep machine learning models are powerful precisely because they can take arbitrary inputs and learn a mapping to the desired outputs.

Deep Learning in Epidemiology

The SIR model divides a population into three categories — Susceptible, Infected, and Recovered — and attempts to define the transitions among them. In the past, identifying these categories and their transitions accurately was extremely difficult.

Deep learning has demonstrated its ability to handle exactly this kind of complex disease dynamics and multi-dimensional data — data that traditional compartmental models and statistical models struggle to capture. Deep learning can analyze and understand the intricate patterns of disease transmission, offering a finer-grained and more dynamic analysis of spread than traditional methods.

COVID Modeling

Epidemiological modeling depends on both time and space: tomorrow’s case count depends on yesterday’s case count and on today’s counts among neighbors. It is therefore a multimodal problem — and also a graph problem.

The team models it with a layered graph.

In the figure above, each node represents a particular time and place, and its own features are the case count and internal mobility data.

The graph is modeled as 150 layers, each layer corresponding to one point in time. Edges within a layer capture spatial properties, with weights representing travel distance; edges between layers capture temporal properties, with weights based on the time elapsed between the nodes they connect (the longer the interval, the lower the weight — an inverse relationship).

This design lets the model capture both dynamics over time (through the temporal edges) and connections across space (through the spatial edges), leading to a better understanding and prediction of how COVID-19 spreads.

This example illustrates a broader point: a major advantage of graph data is the ability to bring context into the analysis — when analyzing a node, its surrounding neighbors can serve as additional sources of information.

GCNs exploit exactly this: by learning the complex relationships between a node and its neighborhood, they improve both the accuracy and the efficiency of analysis and prediction. Such a model considers not only the attributes of an individual node but also its position in the graph and the influence of the nodes around it.

Privacy-Preserving Computation

Privacy is a fundamental concern when analyzing user data, and graph data (social network graphs, transaction graphs, and so on) is no exception. The talk raises two application scenarios around graph data privacy:

Using graph mining (graph clustering) to improve user privacy: can graph mining techniques, and graph clustering in particular, strengthen user privacy? The key question is whether we can use these techniques to identify and protect the sensitive parts of user data, thereby raising the level of privacy protection.
Protecting user privacy in social network applications: how do we protect user privacy in social network applications? Social graphs typically contain a great deal of sensitive information about personal relationships and interactions. The key question is how to design and implement mechanisms that protect this data from unauthorized access or misuse, while still permitting legitimate and beneficial data analysis and social interaction.

These two scenarios reflect the dual challenge of privacy protection for graph data: on one hand we want to exploit the structure and properties of graph data to strengthen user privacy, and on the other we must ensure the data is not abused and that users are shielded from privacy violations.

On the privacy-preserving computation front, one Google effort is FLoC — Federated Learning of Cohorts.

FLoC

FLoC (Federated Learning of Cohorts) is a privacy-preserving web advertising technology and part of Chrome’s Privacy Sandbox initiative, whose goal is to phase out the use of third-party cookies.

Its basic principle is to take a user’s browsing history as input and replace identifiable third-party cookies with anonymous cookies shared by many users, creating FLoCs (cohorts) that contain at least k users with similar browsing interests. In this way, advertisers can target cohorts with similar interests rather than serving ads based on the specific behavioral data of individual users. This reduces the intrusion on personal privacy while still allowing personalized ad targeting.

An important question in this project is how to determine cluster size. In other words, the clustering process must ensure that every cluster reaches some minimum size, since individuals are harder to single out within larger clusters.

Through experiments, the authors show that Affinity maintains high cosine similarity while also ensuring a high degree of anonymity.

On-Device Public-Private Graph Model

The On-Device Public-Private Graph Model is a model for data processing and analysis in recommender systems. It is designed to perform complex computation on a user’s personal device without leaking the user’s private information. Its focus is on combining public and private data in the computation.

Realizing this model comes down to one question: “Can we keep all private data and contact information on the user’s device, without sacrificing any privacy, and still solve important machine learning problems?”

The approach proposed by the Google team is shown in the figure below.

Users can voluntarily choose which portion to keep locally as private data; the non-private portion is uploaded to the cloud and forms the public data available to the server.

As the figure above shows, the workflow consists of two steps:

The public data in the cloud sends a summary to individual users.
Individual users can exchange public data among themselves.

Clustering and Causal Inference

Causal inference is a branch of statistics that attempts to establish cause-and-effect relationships. Its theory is widely applied in randomized experiments such as clinical trials and A/B tests.

Randomized experiments, however, can suffer from interference: the treatment applied to one subject may affect another. In educational research, for instance, applying a teaching method to a group of students may affect not only those students but also their teachers, their classmates, and the overall learning environment of the school, thereby confounding the results.

To come as close as possible to the “all treated” and “all control” conditions, randomized experiments assign subjects to the treatment or control group by cluster. Returning to the example above: if a study aims to evaluate the effect of an educational intervention that could influence the learning atmosphere of an entire class, the researcher might choose the whole class rather than the individual student as the unit of randomization. That way, all students in a class either receive the intervention or do not, which helps ensure the effect is evaluated at the class level and reduces the problem of interference between individuals.

As the figure above shows, if we take individuals as the experimental units, the units influence one another — with more than ten such interactions. To address this, we take clusters as the experimental units instead. As shown on the right, in that example the number of interactions drops to just four.

How Do We Supervise AI Smarter Than Us? | Weak-to-Strong Generalization

Fri, 15 Dec 2023 00:00:00 +0000

Preface

On December 14, 2023, OpenAI released the paper Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision, accompanied by a blog post on their website, Weak-to-strong generalization. This post offers a brief analysis of that work.

The paper asks: AI is growing more capable by the day, approaching and in places already surpassing human ability, and supervising such superhuman AI is becoming very hard for us. So how should we supervise an AI that is stronger than we are?

The research team studies the question through a simple demo and finds that a GPT-2-level model can elicit most of GPT-4’s capability, bringing it close to GPT-3.5 — generalizing correctly even on hard problems the small model itself gets wrong.

This may well be a new research direction.

The problem

The authors believe superintelligent AI could arrive within the next decade. Yet our understanding of AI systems today is so shallow that we have no idea how to steer and control superintelligence reliably, which puts humanity at risk.

Imagine using a powerful AI to write hundreds of thousands of lines of code and deploying it to production: if dangerous code is buried somewhere in there, the consequences could be severe.

In this scenario the human is the weak supervisor and the AI is the superhuman system. The core question is how a not-so-smart weak supervisor can supervise a smart superhuman AI.

The method

The authors’ proposal is to have a weak AI supervise a strong one.

As the figure shows, the leftmost panel depicts conventional machine learning, where a human supervises a weak AI; the middle panel shows the hard situation humans now face, supervising an AI more capable than themselves; the right panel shows the authors’ experimental setup, where a weak AI supervises a comparatively strong one.

On one hand, intuition says a powerful AI supervised by a weak one should never end up better than its supervisor, since the weak AI may mislead it. On the other hand, a strong AI already carries formidable latent capability from pretraining, and all the weak AI has to do is draw that knowledge out.

So which is it? Will the strong AI be elicited and realize its full potential, or will it be misled by the incomplete or flawed labels the weak AI provides and end up performing worse?

Note that “strong AI” above refers to a model that has been pretrained but not fine-tuned for the task.

Results and conclusions

As the figure shows, GPT-4 (the strong AI, un-fine-tuned) supervised by GPT-2 (the weak AI) reaches roughly GPT-3.5-level performance. With nothing but weak supervision, the authors elicit most of GPT-4’s capability.

Simple as the experiment is, it shows the approach is viable in practice, at least as a first step. The authors also see encouraging signs elsewhere: bootstrapping and optimal early stopping both look promising.

Simple human supervision such as RLHF may, absent further work, be hard to scale to superhuman AI
Weak-to-strong generalization is feasible

The study is both original and thought-provoking. People are forever thinking about how to make AI more capable, but few consider how to supervise a superhuman AI once it surpasses us. As the paper puts it, “This opens up a new research direction that allows us to directly tackle a central challenge of aligning future superhuman models while making iterative empirical progress today.” This is a new field, with plenty of low-hanging fruit waiting to be picked.

Heuristic Shortest Path Algorithms

Tue, 05 Dec 2023 00:00:00 +0000

Preface

Working on LLM-enhanced graph problems lately has given me an interest in heuristic algorithms on graphs, and a quick search turned up this survey from 2006. The paper is old, but shortest path algorithms have not been a hot research area over the past two decades, so there is little newer work to read; besides, many recent papers on LLM reasoning take their inspiration from classical algorithms. Here, then, is my summary of it.

Introduction

Although standard optimal shortest path algorithms can already solve most shortest path problems, they often fail to meet the requirements of tasks on large-scale graphs with low latency. In an in-vehicle route guidance system, for example, an immediate response is required. In such tasks the result does not necessarily have to be “optimal” — “good enough” will do. Heuristic shortest path algorithms were born of this need.

Over the past six decades, researchers have proposed many heuristic methods for reducing the computation time of shortest path algorithms. This paper reviews the various heuristic shortest path algorithms developed over that period.

The Shortest Path Problem and Optimal Algorithms

Let the directed graph be $G(N,A)$, with $N$ vertices and $A$ directed edges. Let $n = |N|$ denote the number of vertices and $m = |A|$ the number of edges. Let $a=(i,j)\in A$ denote a directed edge from $i$ to $j$, and $c_{i,j}$ the cost of going from $i$ to $j$. A path from origin $o$ to destination $d$ can be defined as $path = (o,j),\ldots,(i,d)$, and its cost is the sum of the costs of all its edges, $cost_{path} = \sum_{(i,j)\in path} c_{i,j}$. The shortest path problem is to find the path that minimizes $cost$, $path = {\arg \min}_{path \in \text{all paths}} (cost_{path})$.

Optimal Algorithms

The shortest path problem (SPP) has been studied for more than 40 years in fields such as computer science and transportation. Because of their computational tractability, most research in this area has focused on developing increasingly efficient optimal algorithms for the problem. Most optimal shortest path algorithms are essentially applications of dynamic programming theory to searching for shortest paths in a graph. The shortest path is found through a recursive decision process from the source node to the destination node.

Most shortest path algorithms follow a standard procedure like the one below:

Initialization:
$\begin{array}{l}i=o;L_{(i)}=0;L_{(j)}=\infty;\forall j\neq i;P_{(i)}=\text{NULL.}\\Q=\{i\};\end{array}$
Node selection:

Select node $i$ and remove it from $Q$.
Node expansion

Scan the edges emanating from $i$. For an edge $\alpha=(i,j)$, if $L_{(i)}+c_{a}<L_{(j)}$ then $L_{(j)}=L_{(i)}+c_{a};P_{(j)}=a$ and insert $j$ into the set $Q$.
Termination check

If $Q = \emptyset $, stop; otherwise, repeat Step 2.

Here, the cost from the origin node $o$ to node $i$ is denoted $L(i)$; $P$ is the list that stores, for each node, the incoming edge on the shortest path tree, so $P(i)$ is the incoming edge on the shortest path to node $i$; and $Q$ is the set of nodes to be scanned, which manages the nodes to be examined during the search.

Label-setting algorithm (LS)

The elements of $Q$ are sorted by their current cost.

One notable feature of this approach is that if only the path from the origin node to a single destination node is needed, the algorithm can terminate as soon as the cost of the destination node has been computed. This mode of operation is usually called a one-to-one search pattern.

Dijkstra’s algorithm is an example.

Label-correcting algorithm (LC)

The elements of $Q$ are filled and scanned while the shortest path tree is being built.

Its defining feature is that it cannot report the shortest path between two nodes until the shortest paths to every node in the network have been determined, a mode of operation known as a one-to-many search pattern.

Bellman-Ford and SPFA are examples.

Computational Performance

Among LC algorithms, the double-ended queue and the threshold list data structures dominate in terms of computational efficiency.

Among LS algorithms, Dial’s bucket implementation and the binary heap data structure are the most efficient.

Heuristic Shortest Path Algorithms

For real-time problems in practice, the optimal shortest path algorithms discussed above are often far too computationally expensive.

This “inefficiency” stems from the fact that these algorithms employ “uninformed” outward search techniques that make no use of prior knowledge about the locations of the origin and destination nodes, the composition of the path, or the structure of the network.

For instance, if the origin node lies in the city center and the destination node lies in the south of the city, an optimal algorithm is just as likely to search for minimum-cost routes north of the origin node as south of it.

Intuitively, the more information the search uses, the more efficient the algorithm can be. Researchers in AI recognized this early on and proposed many heuristics that attempt to reduce the search effort by drawing on additional sources of knowledge.

Heuristic search strategies can generally be divided into four kinds:

Limit the search area
Decompose the search problem
Limit the links searched
Some combination of the above

Next we examine these heuristic search strategies and their application to shortest path search.

Limiting the Search Area

The idea behind the “limit the search area” strategy is to draw on prior knowledge about the shortest path from the origin node to the destination node to confine the search to a certain region.

The resulting search area is much smaller than that of an optimal algorithm operating without prior knowledge.

Pruning

The basic idea here is to limit the search area by “pruning” those intermediate nodes that are very unlikely to lie on the shortest path to the destination node.

In a classical urban transportation network, each link (or road segment) is normally connected only to adjacent nodes (such as intersections), and the travel time on a link usually correlates with its length. This property allows the search area to be confined to a designated region around the origin and destination nodes. Nodes outside this region are assumed to have a small probability of lying on the shortest path and can therefore be excluded during the search without further examination.

The key is to define the search area in a way that effectively reduces computation time while still yielding a good solution.

Some researchers have proposed constraining the search range with the following inequality: $$ L_{(i)}+e_{(i,d)}\leqslant E_{(o,d)}, $$ where $L(i)$ is the current minimum cost from the origin node $o$ to node $i$; $e(i,d)$ is the estimated cost from node $i$ to the destination node $d$; and $E(o,d)$ is an upper bound on the minimum cost from the origin node to the destination node.

Incorporating this method into the optimal LC algorithm requires only a modification to Step 2:

Select node $i$ and remove it from $Q$. If $L_{(i)}+e_{(i,d)}\leqslant E_{(o,d)}$, jump to Step 4.

The efficiency of the branch-pruning algorithm is illustrated in the figure above. The new heuristic strategy shrinks the search area from the circle expanded by the optimal LS algorithm to an ellipse. On an ideal Euclidean grid, the search area of such heuristic algorithms can be as small as 20% of that of the LS algorithm.

The efficiency and accuracy of a branch-pruning shortest path algorithm depend on the quality of the estimation functions $e(i,d)$ and $E(o,d)$. Optimality is clearly preserved only when $e(i,d)$ always stays below the minimum cost from node $i$ to the destination node $d$ while $E(o,d)$ always stays above the minimum cost from the origin node $o$ to the destination node $d$. It is also worth noting that as $e(i,d)$ approaches zero and $E(o,d)$ approaches infinity, the branch-pruning algorithm degenerates into an optimal shortest path algorithm.

In short, with the method above, $e$ should be underestimated and $E$ overestimated.

A*

In the pruning approach, nodes with a low probability of lying on the shortest path are pruned away. The A* algorithm instead keeps these nodes in $Q$ but assigns them a low priority.

A* uses a heuristic evaluation function $F_{(i)}=L_{(i)}+e_{(i,d)}$ as the label of node $i$, where $L(i)$ is the cost of the currently evaluated path from the origin node to node $i$ and $e(i,d)$ is the estimated cost from node $i$ to the destination node $d$. The sum $F$ of the two functions reflects the likelihood that $i$ lies on the shortest path: the lower $F$ is, the more likely $i$ is to appear on the shortest path. Building on this idea, the algorithm performs a best-first search: it maintains a list $Q$ of nodes to be scanned, sorted by their $F$ values, and selects the node with the lowest $F$ value for expansion. The selected node is expanded by visiting its neighbors, which are then inserted into $Q$ in order according to their $F$ values. This process continues until the destination node is selected for expansion. A* therefore bears a resemblance to the LS algorithm.

Here is a recommended article for understanding A*; it is clearly organized and offers interactive pages: https://www.redblobgames.com/pathfinding/a-star/introduction.html

Compared with the LS algorithm, A* uses the evaluation function $F(i)$ rather than $L(i)$ to determine the order of nodes in $Q$.

The main difference lies in Step 3, modified as follows:

Node expansion

Scan the edges emanating from $i$. For an edge $\alpha=(i,j)$, if

$L_{(i)}+c_{ij}+e_{(j,d)}<F_{(j)},$

then

$\begin{array}{l}L_{(j)}=L_{(i)}+c_{ij};F_{(j)}=L_{(i)}+c_{ij}+e_{(i,d)};P_{(j)}=a,\end{array}$

and insert $j$ into the set $Q$.

Because A* is based on the best-first idea, any node satisfying the following inequality will be scanned before the algorithm terminates: $$ L_{(i)}+e_{(i,d)}\leqslant L_{(d)}. $$ Hence, as long as the estimation function never overestimates the cost, the optimal solution can be found.

Decomposing the Search Problem

It is widely recognized that the amount of computation required to solve a general search problem usually grows faster than the size of the problem itself. For example, the computation time needed to find the shortest path from an origin node to a destination node depends on the number of nodes searched before the destination is reached, so the computational effort is a higher-order polynomial function of distance. Consequently, decomposing the original problem into smaller subproblems can dramatically reduce complexity.

This section describes how to realize this strategy using bidirectional search and the subgoal method.

Bidirectional Search Strategy

The bidirectional search strategy attempts to split the search process into two independent processes: one advancing forward from the origin, the other working backward from the destination. When these two search processes meet at some intermediate stage, the solution is found.

As the figure above shows, the algorithm builds shortest path trees outward from the origin and the destination simultaneously, until some stopping condition is met.

The effectiveness of a bidirectional algorithm is influenced by two factors:

The rule for alternating between the forward and backward searches.
When the algorithm stops.

For the former, the most intuitive approach is to alternate evenly. Even alternation, however, is not necessarily the most efficient. The best strategy should find the shortest path while scanning as few nodes as possible.

For the latter, some researchers have proposed the following condition: $$ L_{(i)}^o+L_{(i)}^d\leqslant\min_{j\in N}{L_{(j)}^o}+\min_{j\in N}{L_{(j)}^d} $$ What this expression captures is that, for any node $j$, the minimum cost from $o$ to $j$ plus the minimum cost from $j$ to $d$ is greater than or equal to the shortest distance from $o$ to $d$ passing through $i$. In that case, $i$ must be a node on the shortest path from $o$ to $d$.

The astute reader will quickly notice, however, that this is a rather poor criterion.

Sure enough, researchers have shown that with this stopping criterion the resulting bidirectional search algorithm performs worse than a unidirectional one. It is conjectured that the nodes expanded by the bidirectional search process may grow into almost complete unidirectional trees before the condition is satisfied, rather than meeting “in the middle” between the origin and the destination.

A number of researchers later proposed improvements on this basis, such as bidirectional A* with a modified estimation function, or bidirectional A* using multiple intermediate meeting nodes.

The Subgoal Method

A subgoal can be defined as an intermediate state of the optimal solution to a problem. For shortest path search in a road transportation network, subgoals may be those nodes or links located between the origin and the destination, between which the shortest path needs to be determined. If the location of a subgoal is known in advance, the problem of finding the shortest path from origin to destination can be decomposed into two or more smaller problems.

For example, given a single subgoal node, the original problem can be solved by solving two subproblems: finding the shortest path from the origin to the subgoal node, and finding the shortest path from the subgoal node to the destination.

If a subgoal node is known and lies midway along the shortest path from the origin to the destination, using it will reduce the search area by roughly 50% compared with the LS algorithm, as shown in the figure above. If one of the techniques for limiting the search area is used at the same time, the computational savings will be even greater.

The reason this method works is that if we know $i$ to be an intermediate node on the shortest path from $o$ to $d$, then the cost of the shortest path from $o$ to $d$ must equal the cost of the shortest path from $o$ to $i$ plus the cost of the shortest path from $i$ to $d$.

Limiting the Links Searched

During shortest path search, the main decision at each iteration concerns which edges emanating from each node to scan. In traditional shortest path algorithms, when a node is selected for expansion, all edges out of that node are scanned, no matter how likely they are to lie on the shortest path.

The basic idea of limiting the links searched is to skip edges that have a low probability of lying on the shortest path or of being used in practice. The hierarchical search method discussed in the next section is an effective way to implement it.

Hierarchical Search

The basic idea behind hierarchical search is that to solve a complex problem efficiently, the search process should first concentrate on the essential features of the problem, ignoring lower-level details, and only then fill those details in.

Take a driver looking for a route between two locations: we first find the arterial roads the route requires, and then find the minor roads connecting those arterials. The first step narrows the search to arterial roads alone, and the second narrows it to the minor roads attached to them.

Two questions must be resolved for this method:

How should real-world roads be mapped into a hierarchical graph?
How should transitions between levels be controlled? When should the search switch to the next level?

Freeways and major traffic arteries are designed for long-distance travel, whereas local streets mainly serve local vehicle trips. Road links can therefore conveniently be classified according to their function. Some researchers have proposed using edge length as the criterion, extracting long edges to form the high-level subnetwork and grouping shorter edges and their nodes into the low-level subnetwork. Others have proposed dividing all roads into two levels based on attributes such as speed limit and number of lanes.

One problem with hierarchical search algorithms is that they cannot take shortcuts between two arterial roads — moving from one arterial to another via residential streets, for instance. These algorithms are designed to simplify the search by classifying roads into levels (freeways, arterials, residential streets, and so on) and thereby reducing the number of roads that must be considered, and they usually consider higher-level roads such as freeways before lower-level ones such as residential streets.

Reference

Title: Heuristic shortest path algorithms for transportation applications: State of the art

Institution: University of Waterloo, Chongqing University, University of Nebraska-Lincoln

Authors: L. Fua, D. Sunb, L.R. Rilettc

DOI: https://doi.org/10.1016/j.cor.2005.03.027

Date: 2006.11.01

Submodular Functions and Their Optimization

Tue, 28 Nov 2023 00:00:00 +0000

Preface

Submodular functions are an important concept in combinatorial optimization and discrete mathematics, with wide-ranging applications across computer science, economics, and systems engineering.

In machine learning and data mining, for example, submodular functions are used for feature selection, clustering, and influence maximization. Choosing the subset of features that best represents a dataset is a submodular optimization problem: the diminishing-returns property guarantees that each newly selected feature contributes less information than the last, which helps avoid overfitting.

Submodular functions are also used in graph theory to solve minimum cut and maximum flow problems. The influence maximization problem in network analysis is another classic application — maximizing the spread of information through a social network, say.

From Concavity to Submodularity

Before getting into submodularity, let us first look at what concavity means.

As the figure above shows, a function $f:\mathbb{R}\to\mathbb{R}$ is concave if $f^{\prime}(\mathbf{x})$ is non-increasing.

Analogously, as the figure above shows, $f:{0,1}^n\to\mathbb{R}$ is submodular if $\partial_if(x)=f(x+e_i)-f(x)$ is non-increasing for every $i$. Note that the domain of a submodular function is discrete.

With that intuition in hand, here are two definitions of submodularity:

As shown in the figure above, let $S\subset T,j\notin T$. If $f$ satisfies $$ f(S \cup \{j\}) - f(S) \geq f(T \cup \{j\}) - f(T) $$ then $f$ is said to be submodular.
If $f:2^{[n]}\to\mathbb{R}$ satisfies the following for all $S$ and $T$, then $f$ is submodular. $$ f(S\cup T)+f(S\cap T)\leq f(S)+f(T). $$ That is, for any two nested subsets (say $A\subset B$), adding the same new element to $A$ and to $B$ yields a gain for $A$ that is at least as large as the gain for $B$. This is a form of “diminishing returns.”

Submodular functions can describe phenomena such as “the more you buy, the cheaper it gets,” or economies of scale. Consider a shopping trip: the first item may bring considerable satisfaction, but as the number of items grows, each additional one adds less and less.

Optimizing Submodular Functions

Given a submodular function, how do we find its maximum and its minimum?

Lovász extension

Earlier work (Grötschel-Lovász-Schrijver, 1981; Iwata-Fleischer-Fujishige / Schrijver, 2000) showed that the Lovász extension solves the minimization problem for any submodular function $f:{0,1}^n\to\mathbb{R}$ in polynomial time.

The Lovász extension turns a submodular function ( f ) into a convex function $ f^L $. $$ f^L(x) = \mathbb{E}_{\lambda\in[0,1]}[f(\{i : x_i > \lambda\})] $$ Because the resulting $f^L$ lives on a continuous domain, its minimum is easy to compute, and the minimizer of $f^L(x) $ can be converted back into the minimizer of $f(S) $.

Minimizing a submodular function thus reduces to a convex optimization problem, and is solved as one.

Greedy

Recall the definition of a submodular function given above. Intuitively it resembles a concave function, yet maximizing a submodular function is in fact a classic NP-hard problem! (Max Cut and Max Coverage are concrete examples.)

Analogously, as the figure above shows, $f:\{0,1\}^n\to\mathbb{R}$ is submodular if the discrete derivative $\partial_if(x)=f(x+e_i)-f(x)$ is non-increasing for every $i$.
For background on P, NP, NPC, and NP-hard, see http://www.matrix67.com/blog/archives/105

Does that mean the problem is hopeless? Not at all. Even if we cannot obtain the exact maximum of a submodular function, an approximation to the optimum is perfectly acceptable.

Let us first state the problem: $$ \max_{S\subseteq\mathcal{V},S\in\mathcal{I}}f(S),S\subset T\Rightarrow f(S)\leq f(T) $$ Here $f(\cdot)$ is a submodular function defined on the dataset $\nu$, and $\mathcal{I}$ is a constraint that depends on the specific setting (often the full set cannot be chosen). Note that this time we require $f$ to be monotonically increasing.

This problem can be tackled with a greedy strategy: at each step, add to $S$ the element that maximizes the gain. That is, pick the element $i$ that maximizes $ f(S+i)-f(S)$, while keeping $S$ within the feasible region $\mathcal{I}$.

A 1978 theorem of Nemhauser, Wolsey, and Fisher states that if $f$ is monotone and submodular, the solution found by the greedy algorithm is at least $ (1 - 1/e) $ times the optimum of the problem $\max_{S\subseteq\mathcal{V},S\in\mathcal{I}}:|S| ≤ k$.

Nemhauser, Wolsey, Fisher ’78: https://www.cs.toronto.edu/%7Eeidan/papers/submod-max.pdf

For the proof see https://zhuanlan.zhihu.com/p/560699106

We now have a basic grasp of how to solve submodular problems, but the scheme above still leaves several questions open:

For the problem formulated as maximizing $f(S)$, what is the best achievable approximation when $f$ is monotone submodular and $I$ forms a matroid?
How should non-monotone submodular functions be optimized?
What about more general constraints, or combinations of several simple ones?

Later work has proposed further greedy-based methods for submodular optimization.

Multilinear Relaxation

Calinescu, Chekuri, Pál, and Vondrák (’07) turned the discrete optimization problem into a continuous one with the multilinear relaxation of a submodular function.

The multilinear extension $F(x) $ is defined as follows: $$ F(x) = \mathbb{E}[f(\hat{x})] $$ where $\hat{x}$ is obtained by randomly rounding each $x_i$ to 0 or 1 with probability $x_i$.

The multilinear relaxation has the following properties:

$F(x)$ is neither globally concave nor globally convex.
The second derivative $\frac{\partial^2 F}{\partial x_i^2}$ of $F$ is zero.
If $\vec{d}$ is non-negative, then $F(x + \lambda \vec{d})$ is a concave function of $\lambda$.

Once the discrete problem has been made continuous, we can find an approximate solution by continuous optimization and then round it to the nearest integer solution of the discrete problem.

Continuous Greedy

We seek the maximum of $F(x)$, that is $\max{F(x):x\in P)}$, where $F$ is the multilinear extension of a monotone submodular function.

For each $x \in P$, define $v(x)$ as $$ v(x) = argmax_{v \in P}(v \cdot \nabla F(x)) $$ Define $x(t)$ by:

$x(0) = 0$
$\frac{dx}{dt} = v(x)$

with $t$ ranging over $[0, 1]$, and return $x(1)$.

This method guarantees $x(1) \in P$, with $F(x(1))$ at least $(1-\frac1e)$ times the optimum (OPT).

By the chain rule, $$ \frac{dF}{dt}=\frac{dx}{dt}\cdot\nabla F(x(t))=v(x)\cdot\nabla F(x(t))\geq OPT-F(x(t)). $$ from which we obtain $$ F(x(t))\geq(1-e^{-t})\cdot OPT. $$

References

[1] https://theory.stanford.edu/~jvondrak/data/SIDMA-plenary-talk.pdf

[2] https://zhuanlan.zhihu.com/p/560699106

[3] https://www.cs.toronto.edu/%7Eeidan/papers/submod-max.pdf