Fedora 18 and CS - Linux Gaming - Setting Up Dedicated server

The day i saw CS:Go released worldwide, I thinking of howto play it. Allfull fledged games are for the windoes and Mac community right!! But luckily Steam took abrave step in brining gaming to bare linux.

This was totaly unexpected for guys like me and what more to ask for a linux cult follower. Thank You Steam..You guys bought CS - community gaming pushing many of my friends over the last decade into gaming .

And among them where many who just bought PC's to play CS in network.. So now steam in linux and CS:Source available in Linux it is possible for me to go back to the collage days...Ya connect my friends at midnight and go rampage in de_dust and de_aztec.

To start with you can directly download the steam client from Fedora repository. Steam is available for almost all the linux flavours.one you get Fedora on steam go to Steam marketplace and buy the games. Currently CS-Source is available in Linux and its paid..:(  . But anyway its just Rs1000 or 20$ which is worth expecially when you are using Linux where your OS and infra are free. For heavily misersyou can check

in PTB(Pirate bay ) for pirated CS. I am damn sure you will find one.

Now once installed you are ready to do the gaming. But for those who already know CS game should be thinnking of passing the IPs getting connected and doing game ..But latetly Steam guys have changed the way we connect to play game together. They now want you to host a dedicated server and get all clients to connect to DS to play game in internet. For LAN based games that is class C ip systems you can directly connect from  the game using the command "connect <ip>"

But we are all grown ups not the hostel guys or combine study era where we gather machines and connect and play in closed LANs. And so i had to setup a CS-DS so that we all could join in during the night hours and do gaming. So went on the task of setting up a dedicated Counterstrike Source server.

Setting up dedciated Server

You may be thinking its too much of technicalities for just playing a game. But thats the cost you have to pay for not spending on hardware like playstation or xbox and on os .Even if you are in windows or mac you will have to setup the dedicated server. So no blame on Linux. Or else you should be contemt with playing cstrike-1.6 the old older version!

 

You can download the steam client which is the basic requirement for setting up any steam based games.from the link . I basically followed the instructions in https://developer.valvesoftware.com/wiki/SteamCMD.

Which is the official steam tutorial in setting up a dedicated server. I initially tried all those blog based tutorials the common one using the HLDSUpdatetool. And found that those where old ways.The current way was

downloading the steamcmd tooland installing the game server.

Once installed steamcmd. Run

./steamcmd

steam> login anonymous

steam> force_install_dir <directory where you need to install>

steam> app_update <gameid>validate   (where gameid for Counterstrike course server is 232330)

steam>quit

The update phase may take up  hourse based on your broadband connection as it downloads GBsof data for thegame server..Once installedyou can start the game server by moving to the

installed directory. But before you start thegameyou may have to put theserver.cfg filewhich configures your server. You can go and get the server.cfgfile from the link . Dontforget to enable the bots!!

Now you can start the game server by using the command

./srcds_run -console -debug -game cstrike -port 27015  +map de_dust2 +maxplayer

Once started you can connect from the CS client using the command connect ~connect <ip>

Some troubleshooting tips:

See to it you have a dedicated external ip for your internet connection. - ipconfig and see the various ips of your interfaces and note down the external ip. now use whatsmyip.org to seewhats your ip from external.

if both are sameyou can assure that your internet connection is via a external ip.In 3g connection most cases you wont be having havingan external ip .  you are under a giant NAT based network.Then you cant host game. I found that airtel andidea are using the above technique. But if you have reliance or bsnl 3g you have chances of getting as ecternal ip.

Check that your ports are accessible via link .

If not then caseis that you r modem acts as a router and you will need to port forward b/w your machine and modem.

Follow the instructions at link to setup portforwarding in your router. You will have default HL2 server setting in most router which isa single clink way for configuring your router for portforwarding. or else theportforword.com sontian instruction for setting up for each type of games.

Firewall is another area where you are going to face trouble. I simply disable it wheneveri go for gaming. I shouldbe finding a better method in the mean time.

Now that everythingis setup you can rjoiceplaing th  CS:Source among your friends with you hosting it.If you are goingto connect from the server thats hosting then you can connect with the internal network ip. That

was inmy case.

 

Non-Steam client

Ifyour friends are windows guys there isevery chance that they would be using a pirated CS game downloaded from TPB. Then they may face issues like

Invalid steam client ticket. Validation rejected

Then you may have to configure the server for accepting non-steam client useres which are basically the pirated once. In this case you can patch your server for allowing non-steam client.You can download the latest patch from - http://rev-crew.info/threads/download-tutorial-how-to-use-revemu.150/ .

And follow the instructions mentioned their. Always see to it that you download the latest patch.

 

Final tips#

Finally also see that all the friends that are connecing are using the same same version as steam updates the games frequently. You can find the version from within the game console using the command

~version

Current versions are based on a update on June18 as of July-1-2013.

Now what!! Go play GOGOGOG!!!

 

You can connect to my server - kalips-css

And i go online between 9pm-2am. For guys in and around tvm are invited as the lag should be less than 100ms.

Semantic and GeoSPAQRL

I was looking for best practises in using geospatial queries in SPARQL and came across this .
OGC - OpenGeoSpatial Consortium has bought about standards in GeoSpatial queries and represention. For Semantics the suggested nechanism is using
GeoSPARQL queries. GeoSPARQL queries look like

The OGC GeoSPARQL standard supports representing and querying geospatial data on the Semantic Web. GeoSPARQL defines a vocabulary for representing geospatial data in RDF, and it defines an extension to the SPARQL query language for processing geospatial data.

http://www.w3.org/2001/sw/wiki/GeoSPARQL - W3C page for GeoSPARQL.There is also a jena based implementation available at
http://code.google.com/p/geospatialweb/
Sample use case are defined in - http://geosparql.org/

PREFIX co: <http://www.geonames.org/countries/#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> 
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>

SELECT ?link ?name ?pop  ?lat ?lon
WHERE  {
   ?link gs:within(40.157623 -74.855347 41.077281 -73.586426) .  
   ?link gn:name ?name .  
   ?link gn:population ?pop  .
   ?link geo:lat ?lat .
   ?link geo:long ?lon
}

PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>  
SELECT ?link ?label ?pop  
WHERE  {   
   ?city gn:name "Bristol" .  
   ?link gs:nearby(?city 30) .  
   ?link gn:name ?label .  
   ?link gn:population ?pop  
   FILTER ( xsd:int(?pop) < 50000)
}

I was looking for a bigdata^tm based implementation . However i could not find a native implementation . But i came across useekm a plugin

being developed under opensahara. But using them i have to sacrifice the bigdata^tm stack and should use Sail based approach. I was not willing for this

because i was already using NanoSPARQL server for my queries for scalaility. In their case they have implemented a IndexingSail indexer separatly for

improving the performance of GeoSPARQL server. This was done on the SAIL layer and when geosparql data is added they create the extra index. This could improve

the performance. but i was alittle confused.

So i raised to them in forum - And decided to do some tweaks and make use of useekm Geosparql implementation natively in bigdata^tm . As my queries where mainly like

?link gs:nearby(?city 30) . Then I started out with getting the src of useekm and finding how they added them to bigdata^tm They made use of customFunction in bigdata for

the extra function intergration. When i furthur looked into the implementation i found that the subjects need to be

http://www.opengis.net/ont/geosparql# More details are present in https://dev.opensahara.com/projects/useekm/wiki/IndexingSail.

Sample query supported in useekm
PREFIX geo: <http://www.opengis.net/ont/geosparql#>
PREFIX geof: <http://www.opengis.net/def/function/geosparql/>
SELECT DISTINCT ?result WHERE {
  ?result <http://example.org/geometry> ?geometry.
  FILTER(geof:sfWithin(?geometry, "POLYGON((0 0, 0 1, 2 1, 1 0, 0 0))"^^geo:wktLiteral))
}

Deatils about the supported function in useekm - https://dev.opensahara.com/projects/useekm/wiki/GeoReference .

Internaly in useekm they use - http://www.vividsolutions.com/JTS/JTSHome.htm for doing all spatial computations.

Map-reduce logic

Number of maps are based on the number of splits.
Default split size is that of the block size. So that data is correctly partiioned ad block boundries.
Job - > task splitting is equal to number of splits made.
Hence this cannot be controlled. Except by changing the number of splits

The number max of task that run in a node is by default 2 .
This can be changes at each node by setting the parameter - mapred.tasktracker.map.tasks.maximum

Hence if you have a 4 core machine you can force the hadoop to run more than 2 task in a node.
And also its applicable if you are not running any reduce task.
The number of reduce task to be run can be set to zero by setting  job.setNumReduceTasks(0);

Also if in your cluster you have one node a VM with a single core. you can set the max
map task to be run on it to 1or 2 by again setting the - mapred.tasktracker.map.tasks.maximum
in the mapred-site.xml of that node.

The default timeout interval for jobtrackers waiting for job completion is 600s this can be reset by adding
    <property>
          <name>mapred.task.timeout</name>
          <value>3600000</value> <!--1hr -->
    </property>

to mapred-site.xml

json custom and conditional deserializing

Problem : Need to conditionaly deserialize to string or object based on the incoming json feed :

    @JsonDeserialize(using = LocationDeserializer.class)
    public void setLocation(Location location) {

And the custom Deserializer looks Like

@Override
    public Location deserialize(JsonParser jp, DeserializationContext ctxt)
            throws IOException, JsonProcessingException {
        ObjectMapper mapper = new ObjectMapper();
        mapper.setDeserializationConfig(ctxt.getConfig());
        jp.setCodec(mapper);
        String city = null;
        Location location = null;
        if(jp.getCurrentToken() == JsonToken.VALUE_STRING) {
           city = jp.readValueAs(String.class);
           location = new Location();
           location.setCity(city);
        } else {
           location = jp.readValueAs(Location.class);
        }
        return location;
    }

Hbase

start hbase using -  start-hbase.sh
get hbase shell - bin/hbase shell
hbase(main):001:0> status
1 servers, 0 dead, 5.0000 average load

hbase(main):004:0> list
TABLE                                                                                                                                                                          
socialdata                                                                                                                                                                     
test                                                                                                                                                                           
testtable                                                                                                                                                                      
3 row(s) in 0.0380 seconds
hbase(main):002:0> create 'socialdata', 'connection','feeds','personal'
0 row(s) in 0.2930 seconds
create table with name socialdata and columnfamily  - connection feeds personal.
hbase(main):009:0> scan 'socialdata'
display contents of a file
drop 'socialdata' - remove the table
exit - exit hbase

delete a row from hbase
- delete all the columns - delete '','',':'

 delete all contents of a row -
deleteall 'socialdata','facebook:123123123'
Inorder to configure a mapreduce job for a remote hbase cluster we have configuration like :

conf = HBaseConfiguration.create(conf);
conf.set(HConstants.ZOOKEEPER_QUORUM, "athena");
conf.set(HConstants.ZOOKEEPER_CLIENT_PORT, "2222");

//
server configurations

Hbase-site.xml

      hbase.rootdir
      hdfs://******:54310/hbase/sprout
 
      hbase.zookeeper.property.clientPort
      2222

      hbase.cluster.distributed
      true

      hbase.zookeeper.quorum
      athena

      hbase.zookeeper.property.dataDir
      /home/*****/Deploy/zookeeper
Hbase needs zookeper running to manage the cluster of masters and slaves.
ALso sync the system time across the machine -The time deifference shouldnt be above .5 minute
Ubuntu
ntpdate
Inside hbase-env.sh
set - export HBASE_MANAGES_ZK=true
To tell Hbase to manage its own zookeeper ensemble and specify the zookeeper properties in the hbase-site.xml
ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications.

When connecting from Hbase client this zookeeper quorum is used in our case quorum name is athena


Connecting programmaticaly to hbase is easier. You need to add the respectibe hbase-site.xml into your base classpath and then call the following statement
Configuration conf = HBaseConfiguration.create();
That creates the configuration for you.
piping output from hbase to a text file
echo "get '*****data','facebook:123123123'"|hbase shell > test

Solving time synchronisation in VM - virtual box

From the virtualbox menu select "Install Guest Additions" in the Device menu.
Now You will see the CDROM device in your guest machine loaded with a "media containing the installation files for guest additions"
You can select it and install the guest additions in the guest machines
Once installed restart the guest machine.

Now in the Host machien go to the virtaulBox installation folder.
Their you will find a program named VboxManage.
Using this you mention the time sync latency by issuing command from here

VBoxManage guestproperty set <guest-os-name> "VirtualBox/GuestAdd/VBoxService/-timesync-interval" 1000
VBoxManage guestproperty set <guest-os-name> "VirtualBox/GuestAdd/VBoxService/-timesync-set-threshold" 1000

that should keep your vms time sync to the accuracy of 1s!

BIgdata - compressed/zip/gz reading.

my problem was that i was having a freebase dump of 8GB. Which is gz format.

But when i explode it becomes close to 60GB. that was too large to be read into the hdfs sequentialy.



idealy i needed hadoop to read in in zip format itself . A little google search took me here -

http://blog.cloudera.com/blog/2009/11/hadoop-at-twitter-part-1-splittable-lzo-compression/



I found a personal blog of - https://github.com/kevinweil/hadoop-lzo very useful. And i followed his blog completely.

I got the code from Git. And did

ant compile-java

ant compile-native



And then i copied the native files within my - hadoop-lzo-0.4.15/lib/native/Linux-amd64-64

To the hadoop native lib  folder - hadoop-1.1.0/lib/native/Linux-amd64-64

I also copied the hadoop-lzo-0.4.15.jar to  - hadoop-lzo-0.4.15/lib (* i dbt whether this is needed. anyway did it)



Now you need to add the codec to the Hadoop configuration . For this add the following to the core-site.xml

<!--added for lzo decompress support.need to move to spring-flow -->



<property>

    <name>io.compression.codecs</name>

    <value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.BZip2Codec,com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec</value>

</property>

<property>

    <name>io.compression.codec.lzo.class</name>

    <value>com.hadoop.compression.lzo.LzoCodec</value>

</property>

Now decompressed my gz and compressed it back to lzo format.



gz -d
If your file is corrupt you can do like

      gunzip < file.gz > file.txt
now i did install lzo by
      sudo apt-get install liblzo2-dev

and then i zipped into lzo format

     lzop file.txt

Now i coped the file.lzo into hdfs using :
hadoop -dfs put /lzofiles

Now i ran the the command
./hadoop jar hadoop-lzo-0.4.15.jar  com.hadoop.compression.lzo.LzoIndexer /lzofiles

It gave output like

3/03/07 09:19:04 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library

13/03/07 09:19:04 INFO lzo.LzoCodec: Successfully loaded & initialized native-lzo library [hadoop-lzo rev 6bb1b7f8b9044d8df9b4d2b6641db7658aab3cf8]

13/03/07 09:19:04 INFO lzo.LzoIndexer: LZO Indexing directory /lzofiles...

Initialy i got error like

13/03/07 09:11:46 ERROR lzo.GPLNativeCodeLoader: Could not load native gpl library

java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path

    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1860)

    at java.lang.Runtime.loadLibrary0(Runtime.java:845)

For this i ensured the JAVA_LIBRARY_PATH is set correctly to hadoop native lib folder.

i tried to echo'ed the variable from within the hadoop command file. and ensured thet variable is set. You can set it by putting

export JAVA_LIBRARY_PATH = in hadoop-env.sh

Also ensure that you build those libgplcompression.so files from within your machine. because these are native calls and depend on your machine architecture and os.

Thats it.

Now this is messy i need to fine tune it to suite to the spring batch way of intergration. .I shall do if time permits.till then

Happy coding.


Installing for cloudera is very simple
install the relavant repo and install the libraries

cd /etc/yum.repos.d/ && wget http://archive.cloudera.com/gplextras/redhat/6/x86_64/gplextras/cloudera-gplextras4.repo
yum install hadoop-lzo-cdh4 hadoop-lzo-cdh4-mr1