” 辅导COMP9017程序、data课程程序 写作COMP2017 / COMP9017 Assignment 3Full assignment due: May 31st, 11:59 pm AEST (Week 13 Sunday)Milestone due: May 24th, 11:59 pm AEST (Week 12 Sunday)This assignment is worth 16% of your final assessmentThis is the updated version of Assignment 3. Please see Ed for more details.Task DescriptionIn this assignment, you will create a networked server (JXServer) that sends files to clients inresponse to requests. Your server will support multiple connecting clients simultaneously as well asmultiple simultaneous connections from the same client for increased transfer speeds.Your assignment will also be tested for performance. You will need to correctly manage multipleparallel connections.MilestoneTo ensure you are making progress on this assignment, you need to complete a minimum amount ofwork by May 24th, 11:59 pm AEST (Week 12 Sunday) to receive up to 4 out of 16 total marks.2 out of 4 milestone marks are for passing specified Milestone automatic correctness tests on Ed.By the Milestone deadline, you must also submit in your assignment workspace on Ed, a 300 worddescription of your design of your assignment so far (please use a plain text file). This short descriptionshould outline the general logical flow of your assignment, including initial socket setup andincluding the method by which you are handling multiple connections. You may wish to include anexample scenario of a client request and the flow of data on the network and in your program underthis scenario.You must attend your Week 13 tutorial, where you will have a short 5 minute discussion with yourtutor as to your progress and the design of your assignment submission, with reference to your writtendescription and last submission before the Milestone. See the Submission section at the end of thisdocument for further details.The written description and review by your tutor in your Week 13 tutorial will contribute the other 2out of 4 milestone marks.See the Submission and Mark Breakdown section at the end of this document for further information.1COMP2017 / COMP9017Working on your assignmentStaff may make announcements on Ed ( httpss://edstem.org) regarding any updates or clarifi-cations to the assignment. The Ed resources section will contain a PDF outlining any notes/changes/-corrections to the assignment. You can ask questions on Ed using the assignments category. Pleaseread this assignment description carefully before asking questions. Please ensure that your work isyour own and you do not share any code or solutions with other students.You can work on this assignment using your own computers, lab machines, or Ed. However, it mustcompile and run on Ed and this will determine the grade. It is important that you continually backup your assignment files onto your own machine, flash drives, external hard drives and cloud storageproviders, ensuring that it is private, and only accessible by you. You are encouraged to submityour assignment while you are in the process of completing it to receive feedback and to check forcorrectness of your solution.IntroductionTypically, networking functionality is implemented by kernel space code. On Linux and other Unixlikeoperating systems, the kernel implements the various components required for networking, suchas Wifi capabilities and the Internet Protocol. This is made available to userspace programs throughspecial system calls, which interact with an abstraction known as a socket.Sockets represent a connection to another endpoint on a network, but are analogous to file handlesand are also described by a file descriptor. Reading from or writing to a socket corresponds toreceiving or sending data over the network. However, as there are specialised operations that need tobe performed with sockets, such as accepting or making connections to network destinations, thereare special system calls that are generally used.A full discussion of network stacks and protocols is out of scope for this unit. For this assignment,your server will use standard Transmission Control Protocol (TCP) connections on top of the InternetProtocol, version 4 (IPv4). This is a standard networking protocol combination for ubiquitous servicessuch as HTTP which we use on the World Wide Web.The combination of TCP and IP(v4), as implemented and exposed to userspace by the kernel, permitsthe formation of network connections. We will not be using any other version of IP in this assignment.A client initiates a connection to a server. Both client and server endpoints are defined by an IPaddress, which in IPv4 is a 32 bit unsigned integer, as well as a TCP port number, which is a 16bit unsigned integer. Once the server accepts the connection, the endpoints exchange data to setup areliable connection. Data that userspace programs send into the socket, at either end of the connection,is delivered and made available for receiving by the userspace application at the other endpoint. TCPguarantees that data is delivered reliably and in order over varying network conditions. It also permitseither endpoint to send data simultaneously to each other. However, it treats data as a continuousstream rather than discrete messages; that is for example if an endpoint sends 5 bytes then 10 bytes,the other endpoint will be able to read the data as it is received, with no indication that there wasoriginally a break in sending 5 bytes in.Your server is responsible for creating a listening TCP socket. This means that it waits for inboundTCP connections from clients.Page 2 of 13COMP2017 / COMP9017In software development, you will often be required to program against third party APIs and libraries.To practice this skill, for this assignment you will need to refer to the manpages for system callsmentioned to determine how to employ them, though guidance will be provided in this specification.To create your TCP socket, you need to use socket(2). For the domain argument, please useAF_INET. For the type argument, please use SOCK_STREAM. You can leave the protocol argument as0.You then need to bind your TCP socket to an address using bind(2). This assigns an IP address andTCP port to your end of the socket.You may have seen IPv4 addresses represented in dotted quad notation, such as 192.0.2.1, whichis simply 4 8-bit integers extracted in order from the 32-bit address. You may wish to use inet_aton(3)to convert from dotted quad notation to the 32-bit integer representation, and inet_ntoa(3) to convertin the other direction.Next, you need to specify that your TCP socket will be listening for incoming connections, usinglisten(2).Finally, you will wait for inbound connections on your socket using accept(2). The kernel willqueue for your program connections to the IP address and TCP port combination that you bind(2) to.When accept(2) returns, it creates a new socket which allows your program to communicate withthis particular accepted connection. Your original socket remains listening for further connectionsthat can be accepted with accept(2).Once you have a connected socket, you can send data to the other endpoint using send(2), and youcan receive any data the other endpoint has sent using recv(2).To close a socket, preventing further communication on it, you can use shutdown(2) and/or close(2).This is a very basic sequence of system calls. To support multiple simultaneous connections, youmay wish to use concurrent programming, such as threads or processes. You may also wish to usemechanisms by which you can be informed when a connection is waiting or data can be read. Theseinclude select(2) and poll(2).Please note the distinction between host and network byte order. Network protocols, including the onein this assignment, generally send data such as integers in big-endian (network byte order), whereasyour host system generally stores them as little-endian. To help you convert between byte orderings,please review the manpages byteorder(3), bswap(3) and/or endian(3). For the avoidance ofdoubt, where applicable, data in this assignment is to be sent as big-endian on the network.TaskYou will write your server in C, compiling to a single binary executable. It will accept exactly 1command line argument as follows:./server configuration fileThe argument configuration file will be a path to a binary configuration file. The data ofthis file will be arranged in the following layout from the start: 4 bytes – the IPv4 address your server will listen on, in network byte orderPage 3 of 13COMP2017 / COMP9017 2 bytes – representing the TCP port number your server will listen on, in network byte order All remaining bytes until the end of the file – ASCII representing the relative or absolute pathto the directory (the target directory), from which your server will offer files to clients. Thiswill not be NULL terminated.The configuration file is binary, not plaintext, and does not have any delimiters or terminators (includingnewlines).The full contents of an example configuration file, as a hexdump, are shown below:c0 00 02 01 16 2e 2f 74 65 73 74 69 6e 67The file is explained in order below: c0 00 02 01 – 4 bytes which represent the IPv4 address 192.0.2.1 in network byte order 16 2e – 2 bytes which represent the port number 5678 in network byte order 2f 74 65 73 74 69 6e 67 – 8 bytes of ASCII representing the string /testing which isthe path of the target directoryYour server will listen for and accept connections from clients. Upon a successful connection, clientswill send a number of possible requests, described in the below format. Please note that all integerfields are sent in network byte order. Your server should not send any data except in response to aclient request.When a client is finished making requests on a given connection, they will shutdown(2) the connection.When you detect this (refer to recv(2)), you should close that socket and clean up anyassociated data in your program.The one exception to this is the client may send a Shutdown command (see section below).All client requests and server responses consist of one or more structured series of bytes, calledmessages. Each message will contain the following fields in the below format: 1 byte – Message header; this describes details about the message as follows. First 4 bits – Type digit: A single hexadecimal digit that defines the type of request orresponse. It is unique for different types of messages. 5th bit – Compression bit: If this bit is 1, it indicates that the payload is compressed (seeCompression section below). Otherwise, the payload is to be read as is. 6th bit – Requires compression bit: If this bit is 1 in a request message, it indicates thatall server responses (except for error responses) must be compressed (see Compressionsection below). If it is 0, server response compression is optional. It has no meaning in aresponse message. 7th and 8th bits – padding bits, which should all be 0. 8 bytes – Payload length; unsigned integer representing the length of the variable payload inbytes (in network byte order).Page 4 of 13COMP2017 / COMP9017 Variable byte payload – a sequence of bytes, with length equal to the length field describedpreviously. It will have different meanings depending on the type of request/response message.The full contents of an example message, as a hexdump, are shown below. Note that it is a validexample of a message in this network protocol, but does not correspond to any meaningful request orresponse you actually have to implement in this assignment.d8 00 00 00 00 00 00 00 07 ab ab ab ab ab ab abThis message is explained in order below: d8 – 1 byte message header. This is represented by the binary 11011000. 1101 – 4 bits representing the hexadecimal type 0xD 1 – 1 bit flag indicating this payload is compressed 0 – 1 bit flag which means, if this message is a request, the response does not have to becompressed. If this message is a response, it has no meaning. 00 – 2 bits of 0 padding 00 00 00 00 00 00 00 07 – 8 bytes representing the payload length 7 in network byteorder ab ab ab ab ab ab ab – the 7 byte payloadOn any single connection, clients will only send one request at a time, before waiting for the appropriateresponse. That is, after sending a request, the client will wait for the server to send a completeresponse before sending the next request. For some requests, the server will need to send multipleresponse messages. An example is file retrieval, where the server may need to send a file to the clientsplit over many response messages. In this case, the client will wait until all appropriate responsemessages are received before sending the next request, if any.Error functionalityIf you receive a client request with invalid (unknown) type field, your server is to send back a responsewith type digit 0xf, with no payload (payload length field 0), and then close the connection. Youshould also send this error message if there are any other errors that arise in requests with validrequest type fields. You are to also send back this error response if you receive a client request with atype field that must only be used in server response messages. Error messages are never compressed,even if the original request indicated compression is required.Echo functionalityClients may request an echo. The request type digit will be 0x0. There will be an arbitrary sequenceof bytes in the payload of the request.In response to this request, your server is expected to send back a response with type 0x1. Thepayload of your response should contain the same payload you received in the request. Note that ifPage 5 of 13COMP2017 / COMP9017the original request requires compression, then you need to compress the payload before returningit in the response. However, after decompression, the payload should be identical to the one youreceived.Directory listingThe request type will be 0x2. There will be no payload and the payload length field will be 0. Theclient is requesting a list of all files in the servers target directory.In response to this request, your server is expected to send back a response with type 0x3. The payloadof your response should contain the filenames of every regular file in the target directory provided inthe command line arguments to your server. (You do not have to return subdirectories, links, or anyother type of entry other than regular files). The filenames can be returned in an arbitrary order.These filenames are to be sent end to end in the payload, separated by NULL (0x00) bytes. Include aNULL (0x00) byte at the end of the payload. You will need to set the payload length appropriately. Ifthe directory is empty, send a single NULL (0x00) byte as a payload.File size queryThe request type will be 0x4. The request payload will be a NULL terminated string that represents atarget filename, for which the client is requesting the size.In response to this request, your server is expected to send back a response with type 0x5. The payloadof your response should contain the length of the file with the target filename in your target directory,in bytes, represented as a 8-byte unsigned integer in network byte order. If the requested filenamedoes not exist, return an error response message (see Error functionality).Retrieve fileThe request type will be 0x6. This is a request for part or whole of a file in your servers targetdirectory. The payload will consist of the following structure: 4 bytes – an arbitrary sequence of bytes that represents a session ID for this request. Please seebelow for uniqueness requirements. 8 bytes – the starting offset of data, in bytes, that should be retrieved from the file 8 bytes – the length of data, in bytes, that should be retrieved from the file Variable bytes – a NULL terminated string that represents a target filenameThe filename will not contain any relative or absolute path components, you only need to search inthe target directory, and no subdirectories.In response to this request, your server is expected to send back one or more response messages withtype 0x7. Each response of type 0x7 may represent a portion of the requested file data. It is up toyou how many and how large these portions you send are. It is also up to you the order in which youPage 6 of 13COMP2017 / COMP9017send these portions; the start of the file does not need to be sent first, as long as all requested datais eventually received by the client. Different portions which you send corresponding to the sameoriginal request must not overlap in the byte ranges from the target file they contain. Each payloadmust consist of the following structure: 4 bytes – the same session ID as was provided in the original request 8 bytes – a starting offset of data, in bytes, from the target file, that this response contains 8 bytes – the length of data, in bytes, from the target file, that this response contains Variable bytes – the actual data from the target file at the declared offset and length in thisresponseThe client may open several concurrent requests for the same filename on different simultaneousconnections, with the same session ID. If you receive multiple connections with requests for the samefile range with the same session ID, it means you are able to multiplex your file data across thoseconnections; a single requesting client is unifying the data at the other end. If you choose to do this,you need to ensure that across all connections sharing the same session ID, the whole requested rangeof the file is eventually reconstructed. The client may make an extra concurrent connection for a givenfile at any time.In the diagram below, the blue double headed arrows indicate a successful connection. Originally,the client has opened one connection, requesting the file target.txt, with session ID 0x67A5CC30.The client then opens a second connection, requesting the same file. Because the session ID is thesame, the server accepts the connection, and is able to return file data simultaneously over the twoconnections (note that the requested file range must also be the same, but this is not shown in thediagram).Page 7 of 13COMP2017 / COMP9017You do not have to multiplex your file response across multiple connections. If so, for connectionson which you will not be returning data, you can send a response with type 0x7 with empty payload.However, your program must be returning the requested response on at least one connection amongthose the client opens.If you would like to achieve higher performance, you will need to implement multiplexing of yourfile response across multiple connections.It is not valid to receive a request for a different file, or the same file with a different byte range, withthe same session ID as a currently ongoing transfer. If this occurs, you should send an error responsemessage (see Error functionality). However, once the entirety of a file is transferred, the session IDmay be reused for different files or the same file with a different byte range, in subsequent requests.In the below example diagram, the red arrow indicates a failed connection where an error responseshould be sent. The client has an existing connection requesting the file target.txt, with session ID0x67A5CC30. It has attempted to open a new connection requesting the different file otherfile.txt,with the same session ID. This is invalid; however the client is able to make a request for otherfile.txt,shown using the different session ID 0x1200CFBA. The server is then expected to service these tworequests simultaneously.You may receive a request for the same file with a different session ID while that file is being transferredunder a first session ID. This is considered a separate client that requires a separate copy of thePage 8 of 13COMP2017 / COMP9017file and should receive the appropriate response.If you receive any other invalid request, such as the filename not existing, or the requested byte rangebeing invalid for the size of the target file, you must send an error response message (see Errorfunctionality).Shutdown commandThe request type digit will be 0x8 and the payload will be 0 length. Your server does not send anyresponse. Instead, your server will shutdown(2) and close(2) all current connections, and exit,cleaning up/freeing all threads/processes/memory, as required. You are guaranteed there will be nofurther new connections or requests after this command. You are guaranteed a Shutdown commandwill only be sent after your server completes processing all other requests. After all processing hasbeen completed and resources freed, your server should terminate with exit code 0.Lossless compressionFor any message where the compression bit is set in the message header, the variable length payload islosslessly compressed, which means it is encoded in a way that completely retains the original payloadinformation, but aims to reduce the size by applying a compression algorithm to the data. The payloadwill have the following structure in order (note that the sections are not necessarily aligned to bytes): Variable number of bits – Compressed payload. See below for details. It is not necessarilyaligned to a byte boundary. Variable number of bits – padding with 0 bits to the next byte boundary. This ensures that thestructure is aligned to a byte boundary. 1 byte – an unsigned integer representing how many bits of 0 padding were required in theprevious fieldNote that the payload length field of the compressed message will contain the length of the compressedpayload in bytes. Note that the padding in the compressed payload ensures it is aligned towhole bytes in size.Your server may also receive request messages (which may also be compressed) where the Requirescompression bit is set in the message header. This means that any message(s) that your serversends in response to such messages must be compressed. Your server should never set this bit ina response message; it is only valid in request messages. If this bit is not set for a request, then itis up to you whether to compress response(s) to that request. Compression can reduce the amountof data needing to be sent over the network, but requires a processing time tradeoff to compute thecompressed payload.In this assignment, you will apply compression by replacing bytes of uncompressed data with variablelength bit sequences (bit codes). This works by aiming to encode more frequently used bytes toshorter bit codes, and less frequently used bytes to longer codes. Therefore, it is possible to have datawhich does not compress at all, or in fact compresses to a larger size. The compression dictionarydefines the mapping of bytes to bit codes and consists of 256 segments of variable length. EachPage 9 of 13COMP2017 / COMP9017segment corresponds in order to input byte values from 0x00 to 0xFF. Each segment is not necessarilyaligned to a byte boundary. However, at the end of the 256 segments, there is padding with 0 bits tothe next byte boundary. This means the entire compression dictionary is aligned to a byte boundaryoverall. The overall structure follows: 256 of the following segments, with no padding in between (including no padding to byteboundaries): 1 byte – unsigned integer representing the length of this code in bits; this is equal to thelength of the next field in bits. It is not necessarily aligned to a byte boundary. Variable number of bits – the bit code that is used to encode the byte value correspondingto this segment. It is not necessarily aligned to a byte boundary. Variable number of bits – padding with 0 bits to the next byte boundary. This ensures that theentire dictionary is aligned to a byte boundary.You will be provided with a binary file that contains a compression dictionary in the structure describedabove. This binary file will always be called compression.dict and be present in thesame directory as your server executable during testing. The compression dictionary stays the samefor all requests handled each time your server runs. That is, your server only needs to read the fileonce at start up to obtain the dictionary, then use that dictionary to handle all compressed requests orresponses.An example of the start of a compression dictionary is shown below. Note that this only includes afew segments and the full dictionary would include all 256 segments plus 0 bit padding at the end asrequired:04 c0 4f …This data is explained in order below: The first segment corresponds to the byte 0x00 04 – the first byte is the length of the code for 0x00 in bits (i.e. 4 bits) c0 – the binary for this byte is 11000000. The first 4 bits, 1100, is therefore the bit code for0x00. The next 4 bits is the start of the second segment, corresponding to the byte 0x01. Thesize of the entire first segment for byte 0x00 is 12 bits. 4f – the binary for this byte is 01001111. Remember that segments are not necessarily alignedto byte boundaries. Therefore, the byte representing the length of the code for 0x01 is comprisedof the second 4 bits from c0 and the first 4 bits from 4f. The binary is thereforePage 10 of 13COMP2017 / COMP901700000100, which corresponds to a code length of 4 bits as well. This means that the second 4bits of 4f, 1111, is the code for 0x01. The size of the second segment is also 12 bits. Note that segments will vary in size. Dependingon the size of each segment, The boundary of each segment can be at any bit offset within a byte,not necessarily 0 or 4 bits like in this short example.To create the compressed payload, for each byte in the original payload, obtain the segment in thecompression dictionary corresponding to the byte value. In your compressed output, the bit code forthis segment should be output for this input byte.Using just the segments shown in the example, you could compress data containing bytes 0x00 and0x01. For the following example uncompressed data:01 00 00 01 01 00Simply replace each uncompressed Byte with the bit code from the compression dictionary. That is,the compressed binary would be:1111 1100 1100 1111 1111 1100This would result in the final compressed bytes:fc cf fcThe compressed payload is output as is. However, at the end of the compressed payload, there is 0-bitpadding to the nearest byte boundary, and a single byte indicating how much padding there was (seeabove).To decode compressed data, as you read in bits from the compressed payload, simply reverse theprocess by using the compression dictionary to find the original byte values. When you decompress apayload, interpret that payload as per the other functionality of your server, depending on the messagetype digit contained in the message header.For the Requires compression bit, you do not ever compress already compressed data. For example,if you receive an echo request with compressed payload, and Requires compression set, you do notcompress the payload again in your response.As you may note, input bytes are encoded to variable length bit codes (variable length coding). It isguaranteed that your compression dictionary gives you bit codes that are uniquely decodable. Thismeans there is only one way to decode a compressed payload, even though you are not explicitlyinformed where the variable length bit codes start and end.Notes and Hints For performance reasons, test clients will begin to make connections within approximately 1second of your server startup. Please ensure that any setup you do is finished by this point, andyour server is ready to accept connections. If your server is not ready to accept connections,you may receive a Connection refused error from the test client. You may find it helpful to use fixed-width integer types such as uint8_t and uint64_tPage 11 of 13COMP2017 / COMP9017 When manipulating individual bits, You will find it helpful to review bitfields and/or bitwiseoperations. Depending on what your server is sending, the test client may not always be able to immediatelyvalidate the data sent. For example, if you declare a certain payload length, but only send partof it, the client may wait indefinitely for the remainder of the payload to be sent. There is notimeout built into the assignment protocol. This means that you may receive a ran too longerror due to inherent limits in the testing system.Submission and Mark BreakdownSubmit your assignment via Git to Ed for automatic marking. Your server program will be executed,and automatic client programs will connect to it to carry out testing.Your code will be compiled with the following options. Please note you may not use variable lengtharrays.-O0 -Wal”
添加老师微信回复‘’官网 辅导‘’获取专业老师帮助，或点击联系老师1对1在线指导。
