SO_REUSEADDR과 SO_REUSEPORT는 어떻게 다릅니 까?

SO_REUSEADDR과 SO_REUSEPORT는 어떻게 다릅니 까?

---

man pages소켓 옵션에 대한 프로그래머 문서화 SO_REUSEADDR와는 SO_REUSEPORT다른 운영 체제에 대해 서로 다른 종종 매우 혼란이다. 일부 운영 체제에는 옵션이 없습니다 SO_REUSEPORT. 웹은이 주제에 대한 모순되는 정보로 가득 차 있으며 종종 특정 운영 체제의 단일 소켓 구현에만 해당되는 정보를 찾을 수 있습니다. 이는 텍스트에서 명시 적으로 언급되지 않을 수도 있습니다.

그렇다면과 정확히 어떻게 SO_REUSEADDR다른 SO_REUSEPORT가요?

SO_REUSEPORT제한 없는 시스템이 있습니까?

그리고 다른 운영 체제에서 둘 중 하나를 사용하는 경우 예상되는 동작은 정확히 무엇입니까?

---

이식성의 놀라운 세계에 오신 것을 환영합니다. 이 두 옵션을 자세히 분석하고 다른 운영 체제에서이를 처리하는 방법을 자세히 살펴보기 전에 BSD 소켓 구현이 모든 소켓 구현의 어머니라는 점에 유의해야합니다. 기본적으로 다른 모든 시스템은 특정 시점 (또는 적어도 해당 인터페이스)에서 BSD 소켓 구현을 복사 한 다음 자체적으로 발전시키기 시작했습니다. 물론 BSD 소켓 구현도 동시에 발전했기 때문에 나중에 복사 한 시스템은 이전에 복사 한 시스템에서는 부족했던 기능을 갖게되었습니다. BSD 소켓 구현을 이해하는 것은 다른 모든 소켓 구현을 이해하는 열쇠이므로 BSD 시스템에 대한 코드를 작성하는 데 관심이 없더라도 이에 대해 읽어야합니다.

이 두 가지 옵션을 살펴보기 전에 알아야 할 몇 가지 기본 사항이 있습니다. TCP / UDP 연결은 5 가지 값의 튜플로 식별됩니다.

{<protocol>, <src addr>, <src port>, <dest addr>, <dest port>}

이러한 값의 고유 한 조합은 연결을 식별합니다. 결과적으로 두 연결이 동일한 5 개의 값을 가질 수 없습니다. 그렇지 않으면 시스템이 더 이상 이러한 연결을 구별 할 수 없습니다.

소켓의 프로토콜은 socket()함수 로 소켓을 만들 때 설정됩니다 . 소스 주소와 포트는 bind()기능 으로 설정됩니다 . 대상 주소와 포트는 connect()기능 으로 설정됩니다 . UDP는 비 연결 프로토콜이기 때문에 UDP 소켓을 연결하지 않고도 사용할 수 있습니다. 그러나 그것들을 연결할 수 있으며 경우에 따라 코드 및 일반 응용 프로그램 디자인에 매우 유리합니다. 비 연결 모드에서 데이터가 처음 전송 될 때 명시 적으로 바인딩되지 않은 UDP 소켓은 일반적으로 시스템에 의해 자동으로 바인딩됩니다. 바인딩되지 않은 UDP 소켓은 데이터를 수신 할 수 없습니다. 바인딩되지 않은 TCP 소켓의 경우에도 마찬가지이며 연결되기 전에 자동으로 바인딩됩니다.

소켓을 명시 적으로 바인드하면 0"모든 포트"를 의미 하는 port에 바인드 할 수 있습니다. 소켓은 실제로 모든 기존 포트에 바인딩 될 수 없기 때문에 시스템은이 경우 특정 포트 자체를 선택해야합니다 (일반적으로 미리 정의 된 OS 별 소스 포트 범위에서). 소스 주소에 대해 유사한 와일드 카드가 존재하며 "모든 주소"일 수 있습니다 ( 0.0.0.0IPv4 및::IPv6의 경우). 포트의 경우와 달리 소켓은 "모든 로컬 인터페이스의 모든 소스 IP 주소"를 의미하는 "모든 주소"에 실제로 바인딩 될 수 있습니다. 소켓이 나중에 연결되면 소켓을 연결할 수없고 동시에 로컬 IP 주소에 바인딩 할 수 없기 때문에 시스템은 특정 소스 IP 주소를 선택해야합니다. 대상 주소와 라우팅 테이블의 내용에 따라 시스템은 적절한 소스 주소를 선택하고 "임의"바인딩을 선택한 소스 IP 주소에 대한 바인딩으로 바꿉니다.

기본적으로 두 개의 소켓은 소스 주소와 소스 포트의 동일한 조합에 바인딩 될 수 없습니다. 소스 포트가 다른 한 소스 주소는 실제로 관련이 없습니다. socketA에 A:X및 socketB에 바인딩 ( B:Ywhere Aand Bare addresses Xand Yare ports)은 X != Ytrue 를 유지하는 한 항상 가능 합니다. 그러나이 경우에도 true X == Y를 A != B유지 하는 한 바인딩은 여전히 ​​가능 합니다. 예를 socketAFTP 서버 프로그램에 속하며에 바인딩 192.168.0.1:21과 socketB다른 FTP 서버 프로그램에 속하며에 바인딩 10.0.0.1:21, 두 바인딩이 성공합니다. 그러나 소켓은 "모든 주소"에 로컬로 바인딩 될 수 있습니다. 소켓이 바인딩 된 경우0.0.0.0:21, 동시에 모든 기존 로컬 주소에 바인딩되며,이 경우 바인딩하려는 21특정 IP 주소에 관계없이 0.0.0.0모든 기존 로컬 IP 주소와 충돌 하므로 다른 소켓을 port에 바인딩 할 수 없습니다 .

지금까지 말한 것은 모든 주요 운영 체제에서 거의 동일합니다. 주소 재사용이 시작되면 OS에 따라 상황이 달라지기 시작합니다. 위에서 말했듯이 BSD는 모든 소켓 구현의 어머니이기 때문에 BSD부터 시작합니다.

BSD

SO_REUSEADDR

경우 SO_REUSEADDR를 바인딩하기 전에 소켓에서 사용되는 바인딩 다른 소켓과 충돌이없는 한, 소켓이 성공적으로 바인딩 할 수 있습니다 정확히 소스 주소와 포트의 같은 조합. 이제 이전과 어떻게 다른지 궁금 할 것입니다. 키워드는 "정확히"입니다. SO_REUSEADDR주로 충돌을 검색 할 때 와일드 카드 주소 ( "모든 IP 주소")가 처리되는 방식을 변경합니다.

없이 SO_REUSEADDR바인딩 socketA에 0.0.0.0:21바인딩 다음과 socketB하기 192.168.0.1:21(오류와 함께 실패합니다 EADDRINUSE0.0.0.0 수단 "로컬 IP 주소", 따라서 모든 로컬 IP 주소가이 소켓에 의해 사용 간주되며이 포함되어 있기 때문에) 192.168.0.1도. 로 SO_REUSEADDR가 있기 때문에, 성공 0.0.0.0하고 192.168.0.1있습니다 정확히되지 같은 주소, 하나는 모든 로컬 주소에 와일드 카드이고 다른 하나는 매우 특정 지역의 주소입니다. 문 위의 어떤 순서에 관계없이 사실이라고주의 socketA와 socketB결합되어, SO_REUSEADDR그것 없이는 항상 실패하고 SO_REUSEADDR항상 성공할 것입니다.

더 나은 개요를 제공하기 위해 여기에 표를 만들고 가능한 모든 조합을 나열 해 보겠습니다.

SO_REUSEADDR socketA socketB 결과
-------------------------------------------------- -------------------
  ON / OFF 192.168.0.1:21 192.168.0.1:21 오류 (EADDRINUSE)
  ON / OFF 192.168.0.1:21 10.0.0.1:21 확인
  ON / OFF 10.0.0.1:21 192.168.0.1:21 확인
   OFF 0.0.0.0:21 192.168.1.0:21 오류 (EADDRINUSE)
   OFF 192.168.1.0:21 0.0.0.0:21 오류 (EADDRINUSE)
   ON 0.0.0.0:21 192.168.1.0:21 확인
   ON 192.168.1.0:21 0.0.0.0:21 확인
  ON / OFF 0.0.0.0:21 0.0.0.0:21 오류 (EADDRINUSE)

위의 표는에 대해 socketA제공된 주소에 이미 성공적으로 바인딩 된 socketA다음 socketB생성되고 SO_REUSEADDR설정되거나 설정되지 않고 마지막으로에 지정된 주소에 바인딩되었다고 가정합니다 socketB. Result에 대한 바인드 작업의 결과입니다 socketB. 첫 번째 열에라고 표시 ON/OFF되면의 값은 SO_REUSEADDR결과와 관련이 없습니다.

알겠습니다 SO_REUSEADDR. 와일드 카드 주소에 영향을줍니다. 알아두면 좋습니다. 그러나 그것은 그것이 가진 유일한 효과가 아닙니다. 대부분의 사람들 SO_REUSEADDR이 처음에 서버 프로그램에서 사용하는 이유이기도 한 또 다른 잘 알려진 효과가 있습니다 . 이 옵션의 다른 중요한 사용을 위해 우리는 TCP 프로토콜이 어떻게 작동하는지 더 깊게 살펴 봐야합니다.

A socket has a send buffer and if a call to the send() function succeeds, it does not mean that the requested data has actually really been sent out, it only means the data has been added to the send buffer. For UDP sockets, the data is usually sent pretty soon, if not immediately, but for TCP sockets, there can be a relatively long delay between adding data to the send buffer and having the TCP implementation really send that data. As a result, when you close a TCP socket, there may still be pending data in the send buffer, which has not been sent yet but your code considers it as sent, since the send() call succeeded. If the TCP implementation was closing the socket immediately on your request, all of this data would be lost and your code wouldn't even know about that. TCP is said to be a reliable protocol and losing data just like that is not very reliable. That's why a socket that still has data to send will go into a state called TIME_WAIT when you close it. In that state it will wait until all pending data has been successfully sent or until a timeout is hit, in which case the socket is closed forcefully.

The amount of time the kernel will wait before it closes the socket, regardless if it still has data in flight or not, is called the Linger Time. The Linger Time is globally configurable on most systems and by default rather long (two minutes is a common value you will find on many systems). It is also configurable per socket using the socket option SO_LINGER which can be used to make the timeout shorter or longer, and even to disable it completely. Disabling it completely is a very bad idea, though, since closing a TCP socket gracefully is a slightly complex process and involves sending forth and back a couple of packets (as well as resending those packets in case they got lost) and this whole close process is also limited by the Linger Time. If you disable lingering, your socket may not only lose data in flight, it is also always closed forcefully instead of gracefully, which is usually not recommended. The details about how a TCP connection is closed gracefully are beyond the scope of this answer, if you want to learn more about, I recommend you have a look at this page. And even if you disabled lingering with SO_LINGER, if your process dies without explicitly closing the socket, BSD (and possibly other systems) will linger nonetheless, ignoring what you have configured. This will happen for example if your code just calls exit() (pretty common for tiny, simple server programs) or the process is killed by a signal (which includes the possibility that it simply crashes because of an illegal memory access). So there is nothing you can do to make sure a socket will never linger under all circumstances.

The question is, how does the system treat a socket in state TIME_WAIT? If SO_REUSEADDR is not set, a socket in state TIME_WAIT is considered to still be bound to the source address and port and any attempt to bind a new socket to the same address and port will fail until the socket has really been closed, which may take as long as the configured Linger Time. So don't expect that you can rebind the source address of a socket immediately after closing it. In most cases this will fail. However, if SO_REUSEADDR is set for the socket you are trying to bind, another socket bound to the same address and port in state TIME_WAIT is simply ignored, after all its already "half dead", and your socket can bind to exactly the same address without any problem. In that case it plays no role that the other socket may have exactly the same address and port. Note that binding a socket to exactly the same address and port as a dying socket in TIME_WAIT state can have unexpected, and usually undesired, side effects in case the other socket is still "at work", but that is beyond the scope of this answer and fortunately those side effects are rather rare in practice.

There is one final thing you should know about SO_REUSEADDR. Everything written above will work as long as the socket you want to bind to has address reuse enabled. It is not necessary that the other socket, the one which is already bound or is in a TIME_WAIT state, also had this flag set when it was bound. The code that decides if the bind will succeed or fail only inspects the SO_REUSEADDR flag of the socket fed into the bind() call, for all other sockets inspected, this flag is not even looked at.

SO_REUSEPORT

SO_REUSEPORT is what most people would expect SO_REUSEADDR to be. Basically, SO_REUSEPORT allows you to bind an arbitrary number of sockets to exactly the same source address and port as long as all prior bound sockets also had SO_REUSEPORT set before they were bound. If the first socket that is bound to an address and port does not have SO_REUSEPORT set, no other socket can be bound to exactly the same address and port, regardless if this other socket has SO_REUSEPORT set or not, until the first socket releases its binding again. Unlike in case of SO_REUESADDR the code handling SO_REUSEPORT will not only verify that the currently bound socket has SO_REUSEPORT set but it will also verify that the socket with a conflicting address and port had SO_REUSEPORT set when it was bound.

SO_REUSEPORT does not imply SO_REUSEADDR. This means if a socket did not have SO_REUSEPORT set when it was bound and another socket has SO_REUSEPORT set when it is bound to exactly the same address and port, the bind fails, which is expected, but it also fails if the other socket is already dying and is in TIME_WAIT state. To be able to bind a socket to the same addresses and port as another socket in TIME_WAIT state requires either SO_REUSEADDR to be set on that socket or SO_REUSEPORT must have been set on both sockets prior to binding them. Of course it is allowed to set both, SO_REUSEPORT and SO_REUSEADDR, on a socket.

There is not much more to say about SO_REUSEPORT other than that it was added later than SO_REUSEADDR, that's why you will not find it in many socket implementations of other systems, which "forked" the BSD code before this option was added, and that there was no way to bind two sockets to exactly the same socket address in BSD prior to this option.

Connect() Returning EADDRINUSE?

Most people know that bind() may fail with the error EADDRINUSE, however, when you start playing around with address reuse, you may run into the strange situation that connect() fails with that error as well. How can this be? How can a remote address, after all that's what connect adds to a socket, be already in use? Connecting multiple sockets to exactly the same remote address has never been a problem before, so what's going wrong here?

As I said on the very top of my reply, a connection is defined by a tuple of five values, remember? And I also said, that these five values must be unique otherwise the system cannot distinguish two connections any longer, right? Well, with address reuse, you can bind two sockets of the same protocol to the same source address and port. That means three of those five values are already the same for these two sockets. If you now try to connect both of these sockets also to the same destination address and port, you would create two connected sockets, whose tuples are absolutely identical. This cannot work, at least not for TCP connections (UDP connections are no real connections anyway). If data arrived for either one of the two connections, the system could not tell which connection the data belongs to. At least the destination address or destination port must be different for either connection, so that the system has no problem to identify to which connection incoming data belongs to.

So if you bind two sockets of the same protocol to the same source address and port and try to connect them both to the same destination address and port, connect() will actually fail with the error EADDRINUSE for the second socket you try to connect, which means that a socket with an identical tuple of five values is already connected.

Multicast Addresses

Most people ignore the fact that multicast addresses exist, but they do exist. While unicast addresses are used for one-to-one communication, multicast addresses are used for one-to-many communication. Most people got aware of multicast addresses when they learned about IPv6 but multicast addresses also existed in IPv4, even though this feature was never widely used on the public Internet.

The meaning of SO_REUSEADDR changes for multicast addresses as it allows multiple sockets to be bound to exactly the same combination of source multicast address and port. In other words, for multicast addresses SO_REUSEADDR behaves exactly as SO_REUSEPORT for unicast addresses. Actually, the code treats SO_REUSEADDR and SO_REUSEPORT identically for multicast addresses, that means you could say that SO_REUSEADDR implies SO_REUSEPORT for all multicast addresses and the other way round.

FreeBSD/OpenBSD/NetBSD

All these are rather late forks of the original BSD code, that's why they all three offer the same options as BSD and they also behave the same way as in BSD.

macOS (MacOS X)

At its core, macOS is simply a BSD-style UNIX named "Darwin", based on a rather late fork of the BSD code (BSD 4.3), which was then later on even re-synchronized with the (at that time current) FreeBSD 5 code base for the Mac OS 10.3 release, so that Apple could gain full POSIX compliance (macOS is POSIX certified). Despite having a microkernel at its core ("Mach"), the rest of the kernel ("XNU") is basically just a BSD kernel, and that's why macOS offers the same options as BSD and they also behave the same way as in BSD.

iOS / watchOS / tvOS

iOS is just a macOS fork with a slightly modified and trimmed kernel, somewhat stripped down user space toolset and a slightly different default framework set. watchOS and tvOS are iOS forks, that are stripped down even further (especially watchOS). To my best knowledge they all behave exactly as macOS does.

Linux

Linux < 3.9

Prior to Linux 3.9, only the option SO_REUSEADDR existed. This option behaves generally the same as in BSD with two important exceptions:

1. As long as a listening (server) TCP socket is bound to a specific port, the SO_REUSEADDR option is entirely ignored for all sockets targeting that port. Binding a second socket to the same port is only possible if it was also possible in BSD without having SO_REUSEADDR set. E.g. you cannot bind to a wildcard address and then to a more specific one or the other way round, both is possible in BSD if you set SO_REUSEADDR. What you can do is you can bind to the same port and two different non-wildcard addresses, as that's always allowed. In this aspect Linux is more restrictive than BSD.

2. The second exception is that for client sockets, this option behaves exactly like SO_REUSEPORT in BSD, as long as both had this flag set before they were bound. The reason for allowing that was simply that it is important to be able to bind multiple sockets to exactly to the same UDP socket address for various protocols and as there used to be no SO_REUSEPORT prior to 3.9, the behavior of SO_REUSEADDR was altered accordingly to fill that gap. In that aspect Linux is less restrictive than BSD.

Linux >= 3.9

Linux 3.9 added the option SO_REUSEPORT to Linux as well. This option behaves exactly like the option in BSD and allows binding to exactly the same address and port number as long as all sockets have this option set prior to binding them.

Yet, there are still two differences to SO_REUSEPORT on other systems:

1. To prevent "port hijacking", there is one special limitation: All sockets that want to share the same address and port combination must belong to processes that share the same effective user ID! So one user cannot "steal" ports of another user. This is some special magic to somewhat compensate for the missing SO_EXCLBIND/SO_EXCLUSIVEADDRUSE flags.

2. Additionally the kernel performs some "special magic" for SO_REUSEPORT sockets that isn't found in other operating systems: For UDP sockets, it tries to distribute datagrams evenly, for TCP listening sockets, it tries to distribute incoming connect requests (those accepted by calling accept()) evenly across all the sockets that share the same address and port combination. Thus an application can easily open the same port in multiple child processes and then use SO_REUSEPORT to get a very inexpensive load balancing.

Android

Even though the whole Android system is somewhat different from most Linux distributions, at its core works a slightly modified Linux kernel, thus everything that applies to Linux should apply to Android as well.

Windows

Windows only knows the SO_REUSEADDR option, there is no SO_REUSEPORT. Setting SO_REUSEADDR on a socket in Windows behaves like setting SO_REUSEPORT and SO_REUSEADDR on a socket in BSD, with one exception: A socket with SO_REUSEADDR can always bind to exactly the same source address and port as an already bound socket, even if the other socket did not have this option set when it was bound. This behavior is somewhat dangerous because it allows an application "to steal" the connected port of another application. Needless to say, this can have major security implications. Microsoft realized that this might be a problem and thus added another socket option SO_EXCLUSIVEADDRUSE. Setting SO_EXCLUSIVEADDRUSE on a socket makes sure that if the binding succeeds, the combination of source address and port is owned exclusively by this socket and no other socket can bind to them, not even if it has SO_REUSEADDR set.

For even more details on how the flags SO_REUSEADDR and SO_EXCLUSIVEADDRUSE work on Windows, how they influence binding/re-binding, Microsoft kindly provided a table similar to my table near the top of that reply. Just visit this page and scroll down a bit. Actually there are three tables, the first one shows the old behavior (prior Windows 2003), the second one the behavior (Windows 2003 and up) and the third one shows how the behavior changes in Windows 2003 and later if the bind() calls are made by different users.

Solaris

Solaris is the successor of SunOS. SunOS was originally based on a fork of BSD, SunOS 5 and later was based on a fork of SVR4, however SVR4 is a merge of BSD, System V, and Xenix, so up to some degree Solaris is also a BSD fork, and a rather early one. As a result Solaris only knows SO_REUSEADDR, there is no SO_REUSEPORT. The SO_REUSEADDR behaves pretty much the same as it does in BSD. As far as I know there is no way to get the same behavior as SO_REUSEPORT in Solaris, that means it is not possible to bind two sockets to exactly the same address and port.

Similar to Windows, Solaris has an option to give a socket an exclusive binding. This option is named SO_EXCLBIND. If this option is set on a socket prior to binding it, setting SO_REUSEADDR on another socket has no effect if the two sockets are tested for an address conflict. E.g. if socketA is bound to a wildcard address and socketB has SO_REUSEADDR enabled and is bound to a non-wildcard address and the same port as socketA, this bind will normally succeed, unless socketA had SO_EXCLBIND enabled, in which case it will fail regardless the SO_REUSEADDR flag of socketB.

Other Systems

In case your system is not listed above, I wrote a little test program that you can use to find out how your system handles these two options. Also if you think my results are wrong, please first run that program before posting any comments and possibly making false claims.

All that the code requires to build is a bit POSIX API (for the network parts) and a C99 compiler (actually most non-C99 compiler will work as well as long as they offer inttypes.h and stdbool.h; e.g. gcc supported both long before offering full C99 support).

All that the program needs to run is that at least one interface in your system (other than the local interface) has an IP address assigned and that a default route is set which uses that interface. The program will gather that IP address and use it as the second "specific address".

It tests all possible combinations you can think of:

• TCP and UDP protocol
• Normal sockets, listen (server) sockets, multicast sockets
• SO_REUSEADDR set on socket1, socket2, or both sockets
• SO_REUSEPORT set on socket1, socket2, or both sockets
• All address combinations you can make out of 0.0.0.0 (wildcard), 127.0.0.1 (specific address), and the second specific address found at your primary interface (for multicast it's just 224.1.2.3 in all tests)

and prints the results in a nice table. It will also work on systems that don't know SO_REUSEPORT, in which case this option is simply not tested.

What the program cannot easily test is how SO_REUSEADDR acts on sockets in TIME_WAIT state as it's very tricky to force and keep a socket in that state. Fortunately most operating systems seems to simply behave like BSD here and most of the time programmers can simply ignore the existence of that state.

Here's the code (I cannot include it here, answers have a size limit and the code would push this reply over the limit).

참고URL : https://stackoverflow.com/questions/14388706/how-do-so-reuseaddr-and-so-reuseport-differ