CS6650 Building Scalable Distributed Systems
Check this post from Spring 2023 class. here
Sound easy/ confused? Here are some more guides
Any script in the /etc/rc.local file will be run as the last step in booting linux. To start a Java program when you boot the instance, you can put the following command in this file.
java -jar <pathToYourJar>
Reference: here
sysctl -w | grep net.ipv4.tcp_fin_timeout #it's your ports release time default is 60s, you can make it lower like 15s
sysctl -w net.ipv4.tcp_fin_timeout=15s #set the time to 15s
sysctl -w net.ipv4.tcp_tw_recycle=1 #turn on the Quickly restore socket resources default is 0
sysctl -a | grep port_range #check available ports range if the range is lower than your threads, change it
vim /etc/sysctl.confnet.ipv4.ip_local_port_range = 1024 65535 sysctl -p #make parameters effective`
ulimit -n #check your limit
ulimie -n 66535 #the upper bound of the limit, but when you restart the machine the number will be back, you can write to your /etc/profile
for windows users use Powershell as follows:
icacls.exe your_key_name.pem /reset
icacls.exe your_key_name.pem /grant:r “$($env:username):(r)”
icacls.exe your_key_name.pem /inheritance:r
thats it! your keys.pem have same restrisctions as you use chmod 400
To access SQS you have to create a service role in AWS Academy. See docs here
This link offers an alternate method using credentials.
This video is a good source. Basically, you have to check whether 1) the connectivity between your load balancer and your target group is fine; 2) whether your instances in the target group are working fine.
Also, you need your target group to listen on port 8080 using tomcat. This excellent piazza post from wonderful 2022 TA Heng spells it out.
A potentually even easier solution is to choose TCP rather than HTTP as the protocol. Health checks then succeed if the TCP connection request succeeds.
Check to make sure your intellij IDEA is the ultimate version.
Still cannot create a new Servlet in Intellij?
Check this answer Stackoverflow
Always installing the same versions of software/applications/packages as mentioned in the labs unless you are confident to know the differences between the versions to install the latest.
For example, Tomcat version 9 only works with Javax Servlet API version 4.0 but not version 5.0. Apache Reference, Java EE vs Jakarta EE, Tomcat 10 info
One of the common causes of bugs is developing software in a version that is not compatible with the software you used in the deployment environment.
When installing Tomcat, remember to set the log retention policy of your Tomcat server to a maximum of one day. If you don’t set it, Tomcat will retain all the logs of all requests, which eventually eat up all the storage space of your EC2 instance. To set it.
Set the following property in your config/server.xml file for the Tomcat
maxDays=”1”
In the Project Structure menu, right-click on the (e.g.) Lab2WebApp folder (the first one in the Available Elements list). There should be an option to put your dependencies into the artifact’s WEB-INF/lib directory. Once done, apply the changes and rebuild your artifact. You should (hopefully) not run into the ClassNotFoundException anymore,
You should install Idea Ultimate version (community version has limited support for this feature)
If you encounter error similar to “source option 7 is no longer supported, use target 8 or higher”
Check the dependencies/compilation target in the POML file. Look for something like
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
Note for Java version
This normally wont affect your program. But if you want to know how to solve it. check here
For maven, that means adding the SLF4J dependencies in your POM.xml, with ${slf4j.version} being the latest version of slf4j
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>
sudo yum install epel-release
sudo amazon-linux-extras install epel
sudo yum install erlang
sudo yum install rabbitmq-server
sudo rabbitmq-plugins list
sudo rabbitmq-plugins enable rabbitmq_management
sudo systemctl enable rabbitmq-server
sudo systemctl start rabbitmq-server
sudo systemctl stop rabbitmq-server
whereis rabbitmq
sudo chown -R ec2-user: /var/log/rabbitmq
http://(YourDNS):15672/
Note you will be running the BAT (Window Batch File) in the bin \ window directory
Example: C:\kafka\bin\windows
First way - write your own java program if you have the time.
Second way - use existing API
You can use the descriptive statistics API to help calculate the statistics required reference
For throughput, typical range achieved in past assignments for a single server instance are about few thousand requests handled per second. (2000 - 5/6000). If you use multiple servers with load balancer, you can potentially achieve higher.
If your throughput is below thousand, you probably didnt explore multi threading enough from your client.
To achieve best throughput of up to 5k/6k, there are a lot of other factors you need to look at. One suggestion is try to look for potential bottleneck(s) in your program and think of how to eliminate them.
The latency figure is directly link to the number of threads you used and the throughput you achieved.
Typical round trip time between your local machine to AWS resources can range between 10ms - 100ms.