forked from cloudfoundry/docs-cloudfoundry-concepts
-
Notifications
You must be signed in to change notification settings - Fork 0
/
understand-cf-networking.html.md.erb
177 lines (144 loc) · 7.43 KB
/
understand-cf-networking.html.md.erb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
---
title: Understanding Container-to-Container Networking
owner: Container-to-Container Networking
---
This topic provides an overview of how Container-to-Container Networking works.
The Container-to-Container Networking feature enables app instances to communicate with each other directly.
<%= vars.cf_networking %>
##<a id="architecture"></a> Architecture
### <a id='overview'></a>Overview
Container-to-Container Networking integrates with [Garden-runC](./architecture/garden.html#garden-runc) in a [Diego](./diego/diego-architecture.html) deployment.
The [Container-to-Container Networking BOSH release](https://github.com/cloudfoundry-incubator/cf-networking-release) includes several core components, as well as swappable components.
To understand the components and how they work, see the diagram and tables below.
The diagram highlights <%= vars.product_full %> components in blue and green.
The diagram also highlights swappable components in red.
### <a id='core'></a> Core Components
The Container-to-Container Networking BOSH release includes the following core components:
<table>
<tr>
<th>Part</th>
<th>Function</th>
</tr>
<tr>
<td>Cloud Foundry Command Line Interface (CF CLI) plugin</td>
<td>A plugin that you download to control network access policies between apps.
</tr>
<tr>
<td>Policy Server</td>
<td>A central management node that does the following:
<ul>
<li>Maintains a database of policies for traffic between apps.
The CF CLI plugin calls an API to create or update a record
in the policy database whenever you create or remove a policy.</li>
<li>Exposes a JSON REST API used by the CF CLI plugin</li>
</ul></td>
</tr>
<tr>
<td>Garden External Networker</td>
<td>
A Garden-runC add-on deployed to every Diego cell that does the following:
<ul>
<li>Invokes the CNI plugin component to set up the network for each app</li>
<li>Forwards ports to support incoming connections from the Gorouter, TCP Router, and Diego SSH Proxy.
This keeps apps externally reachable.</li>
<li>Installs outbound whitelist rules to support Application Security Groups (ASGs). </li>
</ul>
</td>
</tr>
</table>
###<a id='swap'></a> Swappable Components
The Container-to-Container Networking BOSH release includes the following swappable components:
<table>
<tr>
<th>Part</th>
<th>Function</th>
</tr>
<tr>
<td>Silk CNI plugin<br></td>
<td>A plugin that provides IP address management and network connectivity to app instances as follows:
<ul>
<li>Uses a shared VXLAN overlay network to assign each container a unique IP address</li>
<li>Installs network interface in container using the Silk VXLAN backend. This is a shared, flat L3 network.</li>
</td>
</ul>
</tr>
<tr>
<td>VXLAN Policy Agent<br></td>
<td>Enforces network policy for traffic between apps as follows:
<ul>
<li>Discovers desired network policies from the Policy Server Internal API</li>
<li>Updates iptables rules on the Diego cell to allow whitelisted inbound traffic</li>
<li>Tags outbound traffic with the unique identifier of the source app using the VXLAN Group-Based Policy (GBP) header</li>
</ul>
</td>
</tr>
</table>
##<a id='app-comm'></a> App Instance Communication
Container-to-Container Networking uses an overlay network to manage communication
between app instances.
Each Diego cell assigns IP addresses to its containers using a /24 subnet range,
which supports up to 250 containers per cell. The subnet range you choose for
your overlay network determines how many Diego cells your network supports.
Using the /16 subnet range, <%= vars.product_full %> supports up to 250 Diego cells.
To support a higher or lower number of Diego cells, choose a different subnet range.
For example, use /12 to support more Diego cells or /24 to support fewer Diego cells.
<p class="note"><strong>NOTE</strong>: The overlay network IP range must not conflict with any other IP addresses in the network. If there is a conflict, the Diego cells cannot reach any endpoint that has a conflicting IP address.</p>
### <a id='without-cf'></a>Without Container-to-Container Networking
The diagram below illustrates how two app instances communicate in a deployment without Container-to-Container Networking enabled.
Traffic from **App A** must route out and back in through the Gorouter,
which restricts performance and the protocol used to send the traffic.
In this scenario, **App B** does not know the real source of the traffic it receives and must trust all inbound traffic.
### <a id='with-cf'></a>With Container-to-Container Networking
The diagram below illustrates how app instances communicate in a deployment with Container-to-Container Networking enabled.
In this example, the operator creates two policies to regulate the flow of traffic between **App A**, **App B**, and **App C**.
* Allow traffic from **App A** to **App B**
* Allow traffic from **App A** to **App C**
If traffic and its direction is not explicitly allowed, it is denied. For example, **App B** cannot send traffic to **App C**.
##<a id="c2cvsasg"></a> Container-to-Container Networking versus ASGs
Both application security groups (ASGs) and Container-to-Container Networking policies affect traffic from app instances.
The following table highlights differences between ASGs and Container-to-Container Networking policies.
<table>
<tr>
<th></th>
<th>ASGs</th>
<th>Container-to-Container Networking Policies</th>
<tr>
<tr>
<th>Policy granularity</th>
<td>From a space to an IP address range</td>
<td>From a source app to a destination app</td>
</tr>
<tr>
<th>Scope</th>
<td>For a space, org, or deployment</td>
<td>For app to app only</td>
</tr>
<tr>
<th>Traffic direction</th>
<td>Outbound control</td>
<td>Policies apply for incoming packets from other app instances</td>
</tr>
<tr>
<th>Source app</th>
<td>Is not known</td>
<td>Is identified because of direct addressability</td>
</tr>
<tr>
<th>Policies take affect</th>
<td>After app restart</td>
<td>Immediately</td>
</tr>
</table>
##<a id="policies"></a> Policies
Enabling Container-to-Container Networking for your deployment allows you to create policies for communication between app instances.
The Container-to-Container Networking feature also provides a unique IP address to each app container and provides direct IP reachability between app instances.
The policies you create specify a source app, destination app, protocol, and port so that app instances can communicate directly
without going through the Gorouter, a load balancer, or a firewall.
Container-to-Container Networking supports UDP and TCP, and you can configure policies for multiple ports.
These policies apply immediately without having to restart the app.
##<a id="alt-stack"></a> Alternative Network Stacks
The BOSH release that contains the Container-to-Container Networking feature is composed of a
pluggable network stack. Advanced users or third-party vendors can integrate a different network stack. For more information about third-party plugins, see the [Container-to-Container Networking BOSH release](https://github.com/cloudfoundry-incubator/cf-networking-release/blob/develop/docs/3rd-party.md) documentation.