live debian usb

[make live debian usb drive]

download the img

$ dd if=binary.img of=/dev/sdx

x can be found out through dmesg

flv to mp3

[flv to mp3]
how do you convert flv to a normal mp3 for listening to your ipod?

ffmpeg -i input.flv -acodec copy -ac 2 -ar 44100 -ab 128 output.mp3

rsync ssh

[what is rsync?]

Rsync is a wonderful little utility that's amazingly easy to set up on your machines. Rather than have a scripted FTP session, or some other form of file transfer script -- rsync copies only the diffs of files that have actually changed, compressed and through ssh if you want to for security. That's a mouthful -- but what it means is:

* Diffs - Only actual changed pieces of files are transferred, rather than the whole file. This makes updates faster, especially over slower links like modems. FTP would transfer the entire file, even if only one byte changed.
* Compression - The tiny pieces of diffs are then compressed on the fly, further saving you file transfer time and reducing the load on the network.
* Secure Shell - The security concious of you out there would like this, and you should all be using it. The stream from rsync is passed through the ssh protocol to encrypt your session instead of rsh, which is also an option (and required if you don't use ssh - enable it in your /etc/inet.d and restart your inet daemon if you disabled it for security).

How Do I Change my Linux Password?

[How Do I Change my Linux Password?]

passwd [your password]

iptables to blacklist an ip

[iptables to blacklist an ip]

#iptables block a ip
sudo iptables -I INPUT -s xxx.xxx.xxx.xxx -j DROP
#iptbales flush, unblock all previous rules
sudo iptables -F

Minimize HTTP Requests

[Minimize HTTP Requests]
80% of the end-user response time is spent on the front-end. Most of this time is tied up in downloading all the components in the page: images, stylesheets, scripts, Flash, etc. Reducing the number of components in turn reduces the number of HTTP requests required to render the page. This is the key to faster pages.

One way to reduce the number of components in the page is to simplify the page's design. But is there a way to build pages with richer content while also achieving fast response times? Here are some techniques for reducing the number of HTTP requests, while still supporting rich page designs.

Combined files are a way to reduce the number of HTTP requests by combining all scripts into a single script, and similarly combining all CSS into a single stylesheet. Combining files is more challenging when the scripts and stylesheets vary from page to page, but making this part of your release process improves response times.

CSS Sprites are the preferred method for reducing the number of image requests. Combine your background images into a single image and use the CSS background-image and background-position properties to display the desired image segment.

Image maps combine multiple images into a single image. The overall size is about the same, but reducing the number of HTTP requests speeds up the page. Image maps only work if the images are contiguous in the page, such as a navigation bar. Defining the coordinates of image maps can be tedious and error prone. Using image maps for navigation is not accessible too, so it's not recommended.

Inline images use the data: URL scheme to embed the image data in the actual page. This can increase the size of your HTML document. Combining inline images into your (cached) stylesheets is a way to reduce HTTP requests and avoid increasing the size of your pages. Inline images are not yet supported across all major browsers.

Reducing the number of HTTP requests in your page is the place to start. This is the most important guideline for improving performance for first time visitors. As described in Tenni Theurer's blog post Browser Cache Usage - Exposed!, 40-60% of daily visitors to your site come in with an empty cache. Making your page fast for these first time visitors is key to a better user experience.

Manipulating innerHtml disables event handlers set with JavaScript

[Manipulating innerHtml disables event handlers set with JavaScript]

If you manipulate innerHtml property of an HTML element with children and you've previously set any event handlers on those children with JavaScript (not with onxxxx attributes), they're gone (as the changing of innerHTML effectively erases all the children and recreates them). If you want to retain children event handlers, you should add new children with DOM calls (node.appendChild).

split shell command into multiple lines

[split shell command into multiple lines]
insert "\" to break up your command

check battery status from command line

[check battery status from command line]

acpi -V

some more extremely useful vim tips

[some more extremely useful vim tips]
% : match brackets {}[]()
Ctrl-N Ctrl-P : word completion in insert mode
Ctrl-X Ctrl-L : Line complete SUPER USEFUL
Ctrl-R Ctrk-W : Pull onto search/command line

vim quick search a word

[vim quick search a word]

in command mode, type <8> that translated to *
that will highlight all the words that under the current cursor

this is something i wish i had known at the very beginning using vim :)

vim show tabs

[vim show tabs]

Vim has a mode that tells you exactly what's in your file. Executing the command

:set list

puts you into this mode. When the display is set into "list mode" all characters print. Tabs show up as "^I" and the end of line shows up as $. So in list mode, our two examples look like:

prog: prog.c$
^Icc -g -o prog prog.c$

malloc vs new

[malloc vs new]
If you are a C programmer, you may be wondering what was wrong with the malloc() function. In C,
malloc() is used to allocate a given number of bytes of memory. For the most part, using malloc() is
simple and straightforward. The malloc() function still exists in C++, but we recommend avoiding it.
The main advantage of new over malloc() is that new doesn’t just allocate memory, it constructs objects.

A similar difference exists between the free() function and the delete function. With free(), the
object”s destructor will not be called. With delete, the destructor will be called and the object will be properly cleaned up.

how to use assert

[how to use assert]

For performance-critical code, runtime checks such as uses of assert can impose a
significant performance penalty. In these cases, you can compile your code with the
NDEBUG macro defined, by using the -DNDEBUG flag on your compiler command line.
With NDEBUG set, appearances of the assert macro will be preprocessed away. with NDEBUG flag on, all assert will be taken off.

stdout, stderr c

[stdout, stderr c]

stdout is buffered, while, stderr is not buffered; data written to stderr goes directly to the console.

while (1) {
printf (“.”);
sleep (1);
}

In this loop, however, the periods do appear once a second:

while (1) {
fprintf (stderr, “.”);
sleep (1);
}

Never call virtual functions during construction or destruction Never call virtual functions during construction or destruction

[Never call virtual functions during construction or destruction]
Don't call virtual functions during construction or destruction, because such calls will never go to a more derived class than that of the currently executing constructor or destructor.

c++ destructors that emit exceptions

[c++ destructors that emit exceptions]

Premature program termination or undefined behavior can result from destructors emitting exceptions even without using containers and arrays. C++ does not like destructors that emit exceptions!

gcc default include path

[gcc default include path]

gcc -v

prefix, postfix, infix operators

[prefix, postfix, infix operators]

Prefix
+ab


Postfix

ab+


Infix

a+b

placement new, overloading new operator

[placement new]

placement new is just a way to to overload the new operator

int flag = 0;

typedef unsigned int size_t;

void operator delete(void* p, int i)
{
flag = 1;
}

void* operator new(size_t s, int i)
{
return new char[s];
}

class A {
public:
A() {throw -37;}
};

int main()
{
try {
A* p = new (1234) A;
}
catch (int i) {
}
if (flag == 0)
return 1;
else
return 0;
}

Pure Virtual Functions

[Pure Virtual Functions]


In this simple example, Base is an abstract class, and Base::f() a pure virtual function (denoted by "= 0"). A class is abstract if it contains at least one pure virtual function. No instances of an abstract class may be created, nor may an abstract class be used as a function argument or return type. Abstract class pointers and references are always legal.

sample abstract base class, which contains at least one pure virtual function, it cannot be instantiated

class Base {
public:
virtual void f(int) = 0;
};

Factory Pattern

[Factory Pattern]

The essence of the Factory Pattern is to "Define an interface for creating an object, but let the subclasses decide which class to instantiate. The Factory method lets a class defer instantiation to subclasses"

Const Member Function

[Const Member Function]
Declaring a member function with the const keyword specifies that the function is a "read-only" function that does not modify the object for which it is called.

This is a common situation where a value is computed by "lazy evaluation." That is, the value is not computed until the first request for it in order to speed things up in the event that the request isn't made at all. In this case, use "Mutable" modifier on the variable you want to modified initially to allow this operation.

Polymorphism

[Polymorphism]
Generally, the ability to appear in many forms. In object-oriented programming, polymorphism refers to a programming language's ability to process objects differently depending on their data type or class. More specifically, it is the ability to redefine methods for derived classes. For example, given a base class shape, polymorphism enables the programmer to define different area methods for any number of derived classes, such as circles, rectangles and triangles. No matter what shape an object is, applying the area method to it will return the correct results. Polymorphism is considered to be a requirement of any true object-oriented programming language (OOPL).

tcp vs udp

[tcp vs udp]

As TCP provides features such as congestion control, it would be the preferred protocol to use. Unfortunately due to the fact that TCP is a reliable service, delays will be introduced whenever a bit error or packet loss occurs. This delay is caused by retransmission of the broken packet, along with any successive packets that may have already been sent. This can be a large source of jitter.

TCP uses a combination of four algorithms to provide congestion control, slow start, congestion avoidance, fast retransmit and fast recovery. These algorithms all use packet loss as an indication of congestion, and all alter the number of packets TCP will send before waiting for acknowledgments of those packets. These alterations affect the bandwidth available and also change delays seen on a link, providing another source of jitter.

Combined, TCP raises jitter to an unacceptable level rendering TCP unusable for real-time services. Voice communication has the advantage of not requiring a completely reliable transport level. The loss of a packet or bit error will often only introduce a click or a minor break into the output.

For these reasons most VoIP applications use UDP for the voice data transmission. UDP is a thin layer on top of IP that provides a way to distinguish among multiple programs running on a single machine. UDP also inherits all of the properties of IP that TCP attempts to hide. UDP is therefore also a packet based, connectionless, best-effort service. It is up to the application to split data into packets, and provide any necessary error checking that is required.

Because of this, UDP allows the fastest and most simple way of transmitting data to the receiver. There is no interference in the stream of data that can be possibly avoided. This provides the way for an application to get as close to meeting real-time constraints as possible.

UDP however provides no congestion control systems. A congested link that is only running TCP will be approximately fair to all users. When UDP data is introduced into this link, there is no requirement for the UDP data rates to back off, forcing the remaining TCP connections to back off even further. This can be though of as UDP data not being a "good citizen''. The aim of this project is to characterise the quantity of this drop off in TCP performance.

end to end argument

[end to end argument]

A design principle first explicitly put forth by Jerome H. Saltzer, David P. Reed, and Dave Clark in their paper "End-to-End Arguments in System Design" (ACM Transactions in Computer Systems, November 1984.) The end-to-end argument is about where to put functionality: at which layer in a networking stack, or in which component of a distributed system.

Boiled down to its essence, the argument says that some properties (like reliability) can only be correctly and completely implemented at the endpoints of communication--- usually the application layer. Thus, providing that function at a lower layer is at best an optimization.

The canonical example provided in the paper is file transfer. A checksum on individual packets of data can't protect the file's integrity, since the data might be corrupted on its way to disk. The only way to get true reliability is to read the file back off disk, do a checksum, and compare with the sender's checksum calculation. If it fails, resend the entire file. Transport layer checksums are just an optimization to let you resend smaller chunks of data.

Sometimes the argument is characterized as "dumb network, smart edges" to contrast with the telephone network, where all the functionality resides in the switches. But the end-to-end argument doesn't say the network must be dumb, just that it shouldn't interfere with higher-layer properties. Anything sufficiently compelling for performance can still be placed in the network.

However, the Worse Is Better argument says more about how the Internet actually works. End-to-end solutions may be technically superior, but in the real world most applications treat TCP's reliability as good enough, most network engineers view firewalls as a useful security tool, and NAT has a lot more popularity than IPv6.

linux init process

[linux init process]
Init is the parent of all processes. Its primary role is to create processes from a script stored in the file /etc/inittab, init process has pid of 1

how to screw up your gnome-terminal font

[how to screw up your gnome-terminal font]
type this and enter

^N

malloc free vs new delete

[malloc free vs new delete]
Advantages of new over malloc
1. new invokes the constructor of the object created. malloc does not.
2. new does not need an explicit typecasting as in the case of malloc
[int *p=(int *)malloc(sizeof(int));]
3. new can be overloaded. malloc cannot be.
4. new does not need you to specify the size of the object being allocated.
[C * c = new C;]

Advantages of delete over malloc
1. delete can be overloaded. free cannot.
2. delete invokes the destructor of the object to be deallocated. free
does not do this.

linux mandatory file locking

[linux mandatory file locking]

to enforce mandatory locking, 2 steps are required

[1] first mount the filesystem with the mand mount option
[2] chmod g+s on the file you want to lock

What can the statement delete this in a destructor be harmful?

[What can the statement delete this in a destructor be harmful?]
delete this, will invoke the destructor, so it will be a recursive call to the destructor
infinitely. So the program hangs here.

but delete this can be used beautifully in destructor like self tree recursive destruction like this

~treeNode()

{

if (this->left) {

delete this->left; // first recursive call to destructor treeNode::~treeNode.

this->left = NULL;

}

if (this->right) {

delete this->right; // second recursive call to destructor treeNode::~treeNode.

this->right = NULL;

}

}

gcc compiler -Wall

[gcc compiler -Wall]


-Wall turns on all the warning when you compile

threads vs process

[threads vs process]

threads are lightweight processes that share the main memory space of the parent process. Because of this, they use fewer resources than multiprocess applications, as a result, they enjoy faster context-switch time. since the overhead of context-swtich is lower

mktemp make temp file

[mktemp make temp file with arbitrary length random file]

mktemp /tmp/myprog.XXXXXXXXXXXX

shell variables

[shell variables]

#
Number of arguments given to current process.
@
Command-line arguments to current process. Inside double quotes, expands to individual arguments.
*
Command-line arguments to current process. Inside double quotes, expands to a single argument.
- (hyphen)
Options given to shell on invocation.
?
Exit status of previous command.
$
Process ID of shell process.
0 (zero)
The name of the shell program.
!
Process ID of last background command. Use this to save process ID numbers for later use with the wait command.

Fast Modulo by power of 2

[Fast Modulo by power of 2]


Fast Modulo by power of 2
Finding the modulus is particularly expensive. This operation can be performed mush faster using the & operator, because
a % n == a & (n-1) where n is power of 2.

Checking for powers of two

[Checking for powers of two]

/* The following code uses bitwise operators to determine
if an unsigned integer, x, is a power of two. If x is a power of two, x is represented in binary
with only a single bit; therefore, subtraction by one removes that bit and flips all the
lower-order bits. The bitwise and

then effectively checks to see if any bit is the
same. If not, then it's a power of two.*/

int powerOfTwo(unsigned int x)
{
return !((x-1) & x);
}

[Checking for powers of two]

/* The following code uses bitwise operators to determine
if an unsigned integer, x, is a power of two. If x is a power of two, x is represented in binary
with only a single bit; therefore, subtraction by one removes that bit and flips all the
lower-order bits. The bitwise and

then effectively checks to see if any bit is the
same. If not, then it's a power of two.*/

int powerOfTwo(unsigned int x)
{
return !((x-1) & x);
}

A function object

[A function object]
A function object, like a smart pointer (see Smart Pointers [42, 145]) is an ordinary class object. Whereas a smart pointer type overloads the -> and * (and possibly ->*) operators to mimic a "pointer on steroids," a function object type overloads the function call operator()

An object that in some way behaves like a function, of course. Typically, that would mean an object of a class that defines the application operator - operator().

class Sum {
int val;
public:
Sum(int i) :val(i) { }
operator int() const { return val; } // extract value

int operator()(int i) { return val+=i; } // application
};

void f(vector v)
{
Sum s = 0; // initial value 0
s = for_each(v.begin(), v.end(), s); // gather the sum of all elements
cout << "the sum is " << s << "\n";

// or even:
cout << "the sum is " << for_each(v.begin(), v.end(), Sum(0)) << "\n";
}

C++ POD type

[C++ POD type]

You might think a POD type is a data type that arrives from outer space wrapped in a green protective covering, but POD stands for Plain Old Data and that's just what a POD type is. The exact definition is rather gnarly (see the C++ ISO standard), but the basic idea is that POD types contain primitive data compatible with C. For example, structs and ints are POD types, but a class with a user-defined constructor or virtual function is not. POD types have no virtual functions, base classes, user-defined constructors, copy constructors, assignment operator, or destructor.
A You might think a POD type is a data type that arrives from outer space wrapped in a green protective covering, but POD stands for Plain Old Data and that's just what a POD type is. The exact definition is rather gnarly (see the C++ ISO standard), but the basic idea is that POD types contain primitive data compatible with C. For example, structs and ints are POD types, but a class with a user-defined constructor or virtual function is not. POD types have no virtual functions, base classes, user-defined constructors, copy constructors, assignment operator, or destructor.
To conceptualize POD types you can copy them by copying their bits.

so you can use memset to clear all the bits for POD type, but not class with user-defined constructor or virtual functions! however caution must be taken

When do you Need Initialization Lists?

[When do you Need Initialization Lists?]
When do you Need Initialization Lists? as opposed to assignment?
No Default Constructor
If you have a field that has no default constructor (or a parent class with no default constructor), you must specify which constructor you wish to use.
References
If you have a field that is a reference, you also must initialize it in the initialization list; since references are immutable they can be initialized only once.


for example with Initialization

class Child
{
public:
Child( std::string foo ) : foo( foo ) { }
private:
std::string foo;
};

with assignment

class Child
{
public:
Child( std::string foo )
{
this->foo = foo;
}
private:
std::string foo;
};

Deadlock in pipes

[Deadlock in pipes]

Deadlock can occur when one process opens one end of a pipe for writing and another process opens the other end of the same pipe for writing.

Mandatory versus advisory locking

[Mandatory versus advisory locking]

Both fcntl and flock offer advisory locking. Advisory locking is locking that requires the cooperation of participating processes. Suppose process A acquires an exclusive lock on the file, with the intent to write it. Suppose process B opens the file with the intent to write it. If process B fails to acquire a lock, there is nothing to prevent it from issuing a write system call and corrupting the process that A is writing. For this reason, advisory locking is sometimes called unenforced locking.

System V (and therefore Solaris) offers mandatory or enforced locking as an option. This option is enabled if mandatory lock permissions are set on a file. Mandatory lock permissions are an overload of the set group ID execution bit (02000 in octal). If the set group ID execution bit is set, and if the group execution bit is not set, then all reads and writes to the file will use enforced locking. So, for example:

% chmod 2644 example
% ls -l example
-rw-r-lr-- 1 mre staff 9 Dec 28 10:52 example

This makes file example readable and writable by the file's owner, and readable by everyone else. The appearance of the l in the first field of the output of the ls command tells you that mandatory locking is enabled. Of course, you can use any combination of read or write permissions for the file's owner, group, and world.

thread creation

[thread creation]

A call to pthread_create returns immediately, and the original thread continues exe-
cuting the instructions following the call. Meanwhile, the new thread begins executing
the thread function. Linux schedules both threads asynchronously, and your program
must not rely on the relative order in which instructions are executed in the two
threads.

non-blocking wait

[non-blocking wait]
How can you be sure that you clean up child processes that
have completed so that you don’t leave zombie processes, which consume system
resources, lying around?
One approach would be for the parent process to call wait3 or wait4 periodically,
to clean up zombie children. Calling wait for this purpose doesn’t work well because,
if no children have terminated, the call will block until one does. However, wait3 and
wait4 take an additional flag parameter, to which you can pass the flag value WNOHANG.
With this flag, the function runs in nonblocking mod

gnome-terminal font screws up

[gnome-terminal font screws up]
when you print some non-alphanumeric characters, the terminal just totally screws up, can't read the fonts, text becomes undreadable

type in the following to switch back to the normal viewing mode

echo [space key] [ctrl+v] [ctrl+o] [return]

add and remove service to init scripts

[add and remove service to init scripts]
Insert links using the defaults

sudo update-rc.d foobar defaults

Example of disabling a service

sudo update-rc.d -f foobar remove

ext3 file system disadvantage

[ext3 file system]
The Ext3 file system is just Ext2 with a journal added to it; it uses tables for storage of information about the files and not a B-tree database (ReiserFs, XFS use B-tree)
therefore, Ext3 is slow when dealing with large volumes or large amounts of data

hard links vs symbolic links

[hard links vs symbolic links]
A symbolic link merely contains a text string that is interpreted and followed by the operating system as a path to another file or directory. It is a file on its own and can exist independently of its target. If a symbolic link is deleted, its target remains unaffected. If the target is moved, renamed or deleted, any symbolic link that used to point to it continues to exist but now points to a non-existing file. Symbolic links pointing to non-existing files are sometimes called orphaned.

Under Unix, symbolic links are created with the same ln shell command as hard links, but they behave very differently from the latter. Hard links are just (possibly multiple) names associated with a file and cannot exist without their associated file. When one hard link is moved, renamed or deleted, any other hard links referring to the same file data remain unaffected. Only when the last hard link to a file is deleted, the file content disappears. Unlike hard links, symbolic links can also point to directories and cross volumes.

The interesting thing about hard links is that there is no difference between the original
file and the link: they are just two names connected to the same inode. The disadvantage of
using them is that hard links must exist on the same device, which is rather limiting. But, if
possible, you should always create a hard link instead of a symbolic link because they are
faster. An important advantage for symbolic link is that it can be used to refer to a file that is anywhere, even on a server on the other side of the world

force core dumps

[force core dumps]
enable core dumps

ulimit -c unlimited

then use one of the signal to kill your process
SIGQUIT 3 Core Quit from keyboard
SIGILL 4 Core Illegal Instruction
SIGABRT 6 Core Abort signal from abort(3)
SIGFPE 8 Core Floating point exception
SIGSEGV 11 Core Invalid memory reference

kill -c pid

then you can see a file name core in your working directory, cool huh

nm list symbols from object files

[nm - list symbols from object files]
nm - list symbols from object files

nm a.out

linux binary dump of a file

[linux binary dump of a file]
we can use xxd in binary format

xxd -b filename

very useful

hex dump hexdump octal dump od

[hex dump hexdump octal dump od]
use hexdump to view the file in hex
use od to view to the file in octal

hexdump filename
od filename

I prefer using xxd, because it also shows the viewable strings on the right, quite useful for debugging

xxd hex dump of a file

[xxd hex dump of a file]
xxd hex dump of a file

xxd filename

fuser show which process opens this file

[fuser show which process opens this file]
fuser show which process opens this file

fuser somefilename

stty tostop stop background jobs that try to write to the terminal

[stty tostop stop background jobs that try to write to the terminal]
stty tostop stop background jobs that try to write to the terminal

stty tostop

skill pkill kill process with selection criteria

[skill pkill kill process with selection criteria]

The skill and pkill commands function like the kill command, except that
they try to match process names instead of a process ID. Treat these commands like
loaded weapons, use with extreme caution

linux ELF magic number

[linux ELF magic number]
To identify an object file, the kernel looks for a signature in the file, typically
called a magic number. ELF files, for example, have a signature in the first 4 bytes
of the file—specifically, the byte 0x7f followed by the string 'ELF'

use this command to check your a.out, assembly output

head -c a.out

linux turnoff malloc overcommit

[linux turnoff malloc overcommit]

To force the kernel to allow allocations based only on the physical storage
that is currently available, change the following

sudo echo 2 > /proc/sys/vm/overcommit_memory

dd create a disk image from a CD-ROM

[use dd command to create a disk image from a CD-ROM]
use dd command to create a disk image from a CD-ROM

dd if=/dev/cdrom of=image.iso bs=2k

apt-get apt-cache search

[apt-cache search]
suppose you want to find a package related to apache2
apt-cache search apache2

apt-cache search apache2

sync commit buffer cache to disk

[commit buffer cache to disk]
when you copy a file to your usb drive, and you take it to other computer, sometimes it's not there. because it's still in the buffer cache!

use sync to commit buffer cache to disk

sync

Debian configure exim4

[Debian configure exim4]
the easiesit way is to use dpkg-reconfigure to configure

sudo dpkg-reconfigure exim4-config

debian touchpad too slow

[Debian Touchpad too slow]

sudo vim /etc/X11/xorg.conf
change the protocl to alps

Section "InputDevice"
Identifier “Synaptics Touchpad”
Driver “synaptics”
Option “SendCoreEvents” “true”
Option “Device” “/dev/psaux”
Option “Protocol” “alps”
Option “HorizScrollDelta” “0″
Option “SHMConfig” “true”
EndSection

linux kernel Spin Locks vs Semaphores

[linux kernel Spin Locks vs Semaphores]

Low overhead locking => Spin lock is preferred
Short lock hold time => Spin lock is preferred
Long lock hold time => Semaphore is preferred
Need to lock from interrupt context => Spin lock is required
Need to sleep while holding lock =>Semaphore is required

linux process vs thread implementations

[linux process vs thread implementations]

Linux has a unique implementation of threads. To the Linux kernel, there is no concept of a thread. Linux implements all threads as standard processes. The Linux kernel does not provide any special scheduling semantics or data structures to represent threads. Instead, a thread is merely a process that shares certain resources with other processes. Each thread has a unique task_struct and appears to the kernel as a normal process

linux /dev/null

[/dev/null]
In Unix-like operating systems, /dev/null or the null device is a special file that discards all data written to it (but reports that the write operation succeeded), and provides no data to any process that reads from it (it returns EOF). In Unix programmer jargon, it may also be called the bit bucket or black hole.

new vs malloc

[new vs malloc]

I think the most important distinction between new and malloc is that new not only allocate memory but also construct object, while malloc cannot construct objects.

Abstract data type

[ADT, Abstract data type]
From Wiki:
In computing, an abstract data type (ADT) is a specification of a set of data and the set of operations that can be performed on the data. Such a data type is abstract in the sense that it is independent of various concrete implementations. The definition can be mathematical, or it can be programmed as an interface. A first class ADT supports the creation of multiple instances of the ADT, and the interface normally provides a constructor, which returns an abstract handle to new data, and several operations, which are functions accepting the abstract handle as an argument

Linux cd dash last directory

[Linux cd dash last directory]

use cd dash, cd -, command to change to the directory you were last in

cd -

Linux VIRT, RES, and SHR

[Linux VIRT, RES, and SHR]
The difference among VIRT, RES, and SHR in top output

VIRT stands for the virtual size of a process, which is the sum of memory it is actually using, memory it has mapped into itself (for instance the video card's RAM for the X server), files on disk that have been mapped into it (most notably shared libraries), and memory shared with other processes. VIRT represents how much memory the program is able to access at the present moment. RES stands for the resident size, which is an accurate representation of how much actual physical memory a process is consuming. (This also corresponds directly to the %MEM column.) This will virtually always be less than the VIRT size, since most programs depend on the C library.

SHR indicates how much of the VIRT size is actually sharable memory or libraries). In the case of libraries, it does not necessarily mean that the entire library is resident. For example, if a program only uses a few functions in a library, the whole library is mapped and will be counted in VIRT and SHR, but only the parts of the library file containing the functions being used will actually be loaded in and be counted under RES.

Linux Swapping

[Linux Swapping]

Swapping
Swapping is the moving of an entire process to and from secondary storage when the main memory is low. Many modern operating systems, including Linux, do not use this approach, mainly because context switches are very expensive. Instead, they use paging. In Linux, swapping is performed at the page level rather than at the process level. The main advantage of swapping is that it expands the process address space that is usable by a process. As the kernel needs to free up memory to make room for new pages, it may need to discard some of the less frequently used or unused pages. Some of the pages cannot be freed up easily because they are not backed by disks. Instead, they have to be copied to a backing store (swap area) and need to be read back from the backing store when needed. One major disadvantage of swapping is speed. Generally, disks are very slow, so swapping should be eliminated whenever possible.

Linux epoll edge vs level

[Edge- Versus Level-Triggered Events]

Edge- Versus Level-Triggered Events
If the EPOLLET value is set in the events field of the event parameter passed to
epoll_ctl( ), the watch on fd is edge-triggered, as opposed to level-triggered.
Consider the following events between a producer and a consumer communicating
over a Unix pipe:
1. The producer writes 1 KB of data onto a pipe.
2. The consumer performs an epoll_wait( ) on the pipe, waiting for the pipe to
contain data, and thus be readable.
With a level-triggered watch, the call to epoll_wait( ) in step 2 will return immedi-
ately, showing that the pipe is ready to read. With an edge-triggered watch, this call
will not return until after step 1 occurs. That is, even if the pipe is readable at the
invocation of epoll_wait( ), the call will not return until the data is written onto the
pipe.
Level-triggered is the default behavior. It is how poll( ) and select( ) behave, and it is
what most developers expect. Edge-triggered behavior requires a different approach to
programming, commonly utilizing nonblocking I/O, and careful checking for EAGAIN.

Linux epoll vs poll vs select

[The Event Poll Interface : Linux epoll]
The Event Poll Interface
Recognizing the limitations of both poll( ) and select( ), the 2.6 Linux kernel* intro-
duced the event poll (epoll) facility. While more complex than the two earlier interfaces,
epoll solves the fundamental performance problem shared by both of them, and adds
several new features.
Both poll( ) and select( ) require the full list of file descrip-
tors to watch on each invocation. The kernel must then walk the list of each file
descriptor to be monitored. When this list grows large—it may contain hundreds or
even thousands of file descriptors—walking the list on each invocation becomes a
scalability bottleneck.

linux print repeating string

[linux print repeating string]

suppose you want to print x 100 times, use the following cool method

printf 'x%.0s' {1..100}

Linux tr translating or replacing characters

[Linux tr translating or replacing characters]

try this simple result to see how tr works

>echo "1234567891" | tr 21 xy
>yx3456789y

what tr does is translate 2 with x and translate 1 with y, pretty cool!

Linux Kernel I/O Elevators

[Linux Kernel I/O Elevators]
I/O Elevators:
An elevator is a queue where I/O requests are ordered by the function of their sector on disk. Two I/O elevators are now available in the 2.6 kernel: anticipatory and deadline. The default mode is anticipatory. Using anticipatory mode, synchronous read operations are scheduled together with a delay of a few milliseconds, anticipating the "next" read operation. This mode should help read performance for commands that require multiple synchronous reads, especially during streamed writes. However, several workloads that seek all over the disk, performing reads and synchronous writes, such as database operations, can actually suffer with anticipatory I/O. For these workloads, the deadline I/O scheduler is better and can deliver up to a 10% performance improvement over the anticipatory scheduler. Select the deadline I/O scheduler by booting with elevator = deadline on the kernel command line.

Linux convert ext2 to ext3

[Linux convert ext2 to ext3]
convert the existing ext2 partition /dev/hda1 to ext3, issue the following command

tune2fs j /dev/hda1

Linux Filesystems

[Linux Filesystems]

[ext3] Easy to upgrade from existing ext2 file system
[ReiserFS] Best performance with small files; fully supported by major enterprise distributions
[XFS] Best performance, especially with large files

Linux Performance Tuning Partition

[Linux Performance Tuning Partition]

[1] create separate partitions for root, swap, /var, /usr, and /home.

[2] Accessing Outer tracks are faster than accessing inner tracks. While lower-numbered partitions are usually allocated at the outer tracks: /dev/hda1 is closer to the drive's outer edge than /dev/hda3, so place partitions that require frequent access first.

[3] The first partition should be the swap partition: this is to optimize memory swap operations.

[4] The second partition should be /var because log entries are frequently written to /var/log.

[5] The third partition should be /usr, because base system utilities and commands are placed in /usr.

[6] The root and /home partitions can reside near the end of the drive.

ubuntu archive tar

[ubuntu archive tar]

archive with tar using this command to create a tar file of the folder myfolder, including all subfolders

tar cvf myfolder.tar myfolder/

to untar this file, just use this following command

tar xvf myfolder.tar

to lists the files in the tar file use this command

tar tvf myfolder.tar

enjoy

Linux system function

[Linux System Function]

int system(const char* cmd);

system is implemented by calling fork, exec, waitpid.

Linux fork

[Linux fork]

An existing process can create a new process by calling the fork, the new process created by fork is the child process.

Interesting to know about this function is, fork function is called once, but return twice, one by the parent, one by the child. the only difference in the return is that the return value in the child is 0, whereas the return value in the parent process will be the process ID of child process that created.

Linux Unbuffered vs buffered I/O

[Linux Unbuffered vs buffered I/O]

In buffered I/O, data is buffered and then sent to the file. In unbuffered I/O, the data is immediately sent to the file.

If you drop a number of paper clips on the floor, you can pick them up in buffered or unbuffered mode. In buffered mode, you use your right hand to pick up a paper clip and transfer it to your left hand. The process is repeated until your left hand is full, and then you dump a handful of paper clips into the box on your desk.

In unbuffered mode, you pick up a paper clip and dump it immediately into the box. There is no left-hand buffer.

In most cases, buffered I/O should be used instead of unbuffered. In unbuffered I/O, each read or write requires a system call. Any call to the operating system is expensive. Buffered I/O minimizes these calls.

Unbuffered I/O should be used only when reading or writing large amounts of binary data or when direct control of a device or file is required.

Linux fork exec waitpid

[Linux fork exec waitpid]

There are 3 primary functions for process control: fork, exec, waitpid.

we call fork to create new process, and call exec to execute command, and call waitpid function to specify which process we want to wait for to finish.

C++ struct vs class

[C++ struct vs class]

They are almost the same, excecpt their names are different and that struct members are public by default, while class members are private by default.

linux awk sum every 5 lines

[linux awk sum every 5 lines]


redirect your program output, to let awk sum the input every 5 lines

./program | awk '{if (NR%6 == 0) sum = $1; else sum +=$1 } {print $1, sum }'

this is cool

awk if statement

[awk if then else]

to filter your program output to even or odd number, redirect your output to awk

./program | awk '{ if ($1 % 2 == 0) print "input is even"; else print "input is odd" }'

you can later do fancier things with awk if you want to, just to show the basic concepts here

Linux Shell For Loop

[Class Linux Shell For Loop]

for (( i = 0 ; i <= 10; i++ ))
do
echo "$i"
done

linux file descriptors for every process

[Default file descriptors for every process]

Unless the process explicitly closes them, every process by convention has at least
three file descriptors open: 0, 1, and 2. File descriptor 0 is standard in (stdin), file
descriptor 1 is standard out (stdout), and file descriptor 2 is standard error (stderr).

Linux Simple Rules Regular Expressions

[Linux Simple Rules Regular Expressions]

• ^pattern searches for pattern at the beginning of a line.
• pattern$ searches for pattern at the end of a line.
• \• pattern>\ searches for pattern at the end of a word.

Linux Viewing Text Files

[Linux Viewing Text Files]

• cat: Displays the contents of a file
• tac: show the contents in an inverse order, just like cat written backwards :)
• tail: Shows the last few lines of a text file
• head: shows the first few lines of a file
• less: advanced file viewer
• more: similar to less, but simpler

Linux Run several commands at the same time

[Run Several Commands at the same time]

suppose you have programA, programB, programC , you want to run them all at the same time,

do this

./programA & ./programB & ./programC &

you will run all 3 programs at the same time

ubuntu download youtube flv file

[ubuntu download youtube flv file]

how to download youtube video to your disk? very simple

sudo apt-get install youtube-dl
youtube-dl [youtube video url]

enjoy

Redirect both output and error messages

[Redirect both output and error messages]

this is very interesting

./myprogram 1> output 2> error

very cool

Bash writing output to the terminal

[Bash writing output to the terminal]

Classic Hello World!

echo Hello World!

There you go :)

Showing All Hidden Files in the Current Directory

[Linux Bash Showing All Hidden Files in the Current Directory]

ls -d .*
ls -d .b*
ls -d .[!.]*

or simply, you can use

ls -a

to show all the files

Linux Add User Example

[Linux Add User Example]

useradd - Adding a new user
Options:

* -d home directory
* -s starting program (shell)
* -p password
* -g (primary group assigned to the users)
* -G (Other groups the user belongs to)
* -m (Create the user's home directory

Example: To add a new user with

* a primary group of users
* a second group powerusers
* starting shell /bin/bash
* password of xxxxxx
* home directory of john
* create home directory
* a login name of john

use the following command

useradd -gusers -Gpowerusers -s/bin/shell -pxxxxxx -d/home/john -m john

Linux Slab Benefits

[Slab Benefits]

Few of the benefits:

- Frequently used data structures tend to be allocated and freed often, so cache them.

- Frequent allocation and deallocation can result in memory fragmentation (the inability to find large contiguous chunks of available memory). To prevent this, the cached free lists are arranged contiguously. Because freed data structures return to the free list, there is no resulting fragmentation.

- The free list provides improved performance during frequent allocation and deallocation because a freed object can be immediately returned to the next allocation.

- If the allocator is aware of concepts such as object size, page size, and total cache size, it can make more intelligent decisions.

Linux Slab Allocator

[Linux Slab]

From Reference:

The slab cache

The slab allocator used in Linux is based on an algorithm first introduced by Jeff Bonwick for the SunOS operating system. Jeff's allocator revolves around object caching. Within a kernel, a considerable amount of memory is allocated for a finite set of objects such as file descriptors and other common structures. Jeff found that the amount of time required to initialize a regular object in the kernel exceeded the amount of time required to allocate and deallocate it. His conclusion was that instead of freeing the memory back to a global pool, he would have the memory remain initialized for its intended purpose. For example, if memory is being allocated for a mutex, the mutex initialization function (mutex_init) need only be performed once when the memory is first allocated for the mutex. Subsequent allocations of the memory need not perform the initialization because it's already in the desired state from the previous deallocation and call to the deconstructor.

Linux kmalloc vs vmalloc

[kmalloc vs vmalloc]


kmalloc() function guarantees that the pages are physically contiguous (RAM).
vmalloc() function only ensures that the pages are contiguous within the virtual address space.

kernel code uses kmalloc() and not vmalloc() to obtain memory. due to performance issues. using vmalloc will more likely to result in TLB thrashing than kmalloc.

Linux TLB

[Linux TLB]

From Reference:

Every time the CPU accesses virtual memory, a virtual address must be translated to the corresponding physical address. Conceptually, this translation requires a page-table walk, and with a three-level page table, three memory accesses would be required. In other words, every virtual access would result in four physical memory accesses. Clearly, if a virtual memory access were four times slower than a physical access, virtual memory would not be very popular! Fortunately, a clever trick removes most of this performance penalty: modern CPUs use a small associative memory to cache the PTEs of recently accessed virtual pages. This memory is called the translation lookaside buffer (TLB).

The TLB works as follows. On a virtual memory access, the CPU searches the TLB for the virtual page number of the page that is being accessed, an operation known as TLB lookup. If a TLB entry is found with a matching virtual page number, a TLB hit occurred and the CPU can go ahead and use the PTE stored in the TLB entry to calculate the target physical address. Now, the reason the TLB makes virtual memory practical is that because it is small—typically on the order of a few dozen entries—it can be built directly into the CPU and it runs at full CPU speed. This means that as long as a translation can be found in the TLB, a virtual access executes just as fast as a physical access. Indeed, modern CPUs often execute faster in virtual memory because the TLB entries indicate whether it is safe to access memory speculatively (e.g., to prefetch instructions).

Linux sum a column with awk

[Linux sum a column with awk]


assume you have a file named file-to-sum having these values

1
2
3
4
5
6
7

cat file-to-sum | awk '{ SUM += $1} END { print SUM }'

will sum the result to

28

C++ Bitwise Operators

[C++ Bitwise Operators]

& -Bitwise AND
| -Bitwise OR
^ -Bitwise exclusive OR
~ -Complement
<< -Shift left
>> -Shift right

Linux Process task stats

[Process task stats]

1 task can have 6 stats

TASK_RUNNING
TASK_INTERRUPTIBLE
TASK_UNINTERRUPTIBLE
TASK_STOPPED
TASK_ZOMBIE
TASK_DEAD

Linux How process is implemented

[Linux Process]

Linux manages the processes in the system by having each process be represented by a task_struct data structure, which is dynamically allocated by the kernel.

Linux proc

[What is /proc]

The /proc file system is a reflection of the system in memory. It can be used as a hierarchal view of the system. The goal of the /proc file system is to provide an easy view of the kernel resources and components. You also can use /proc to view information about the processes that are currently running on the system. A second goal of /proc is to present information about the system in a readable way instead of getting this type of information through API system calls.

Linux Kernel Threads vs Green Threads

[Linux Kernel Threads vs Green Threads]

Overview of Kernel Threads and Green Threads

[Kernel Threads]
- Scheduled by kernel
- managed in kernel space
- have better performance on I/O and context switching operations

[Green Threads]
- scheduled by a Virtual Machine (VM)
- emulate multithreaded environments without relying on any native OS capabilities.
- managed in user space
- outperform kernel threads on thread activation and synchronization

vim how to indent a block

[How to indent a block with vim]

to indent repeatedly

enter command mode, shift-v, then type >, then type . to repeat

to unindent repeatedly

enter command mode, shift-v, then type <, then type . to repeat

very COOL

Linux vim Visual Mode

[3 vim Visual Mode]

v - start Visual mode per character
V - start Visual mode linewise
CTRL-V - start Visual mode blockwise

very useful to edit your document with vim using these 3 modes, blockwise mode is extremely unique.

Linux Grep File Text In Files

[Use Grep To Find Text In Files]

use this grep command

grep -r [pattern] [path]

this will search a pattern in the path recursively

Linux run in background when you log off

[How to run a Unix/Linux command in the background even when you log off nohup]

use the nohup command before your command like this

nohup [your command]&

Linux vim autocomplete

[Linux vim autocomplete for PHP & HTML]


type these command, or add these into your vimrc file

autocmd FileType php set omnifunc=phpcomplete#CompletePHP
autocmd FileType html set omnifunc=htmlcomplete#CompleteTags

when you encounter a word you need autocomplete, use the magic Ctrl-p

Install Ubuntu on Mac Mini

[Install Ubuntu On Mac Mini]

Now that Mac is equipped with Intel Core 2, can we install ubuntu? yes

[1] make yourself a ubuntu install cd
[3] insert the cd into mac, while you can
[2] shut down mac mini
[3] hold down 'c' key, on your keyboard
[4] turn on mac mini

there you go, now you can install ubuntu with ease,

however, remember to hold down c key when you start :)

Linux rm argument list too long workaround

[rm argument list too long]

annoyed by argument list too long error? use this instead

find . -name [pattern] | xargs rm

vim save and load your setting

[vim save and load your settings]

:mkvimrc workspace

you can use this to save your general settings to a file workspace,

then you can manually edit this file, then you can load this file by

:source workspace

a very organized way of re-use your workspace area

Ubuntu Install Trac

[Ubuntu Install The Most Basic Trac]


this will install your trac

sudo apt-get install trac

next you should let trac-admin create a project for you, go to your directory where you want to setup your project

trac-admin MyProjectToTrack initenv

keep hitting , you can always change stuff later,

append this line to your /etc/apache2/apache2.conf

Alias /trac "/usr/share/trac/htdocs"

we are almost done, now choose a port to make your service available

tracd --port 8000 [path to your project you setup previously]

this means now you can manage your project through an online interface from

http://localhost:8000

Linux Find Out Your CPU INFO

[Find Out Your CPU Info]


You can find out details about the cpu of your system by this:

cat /proc/cpuinfo

Symmetric multiprocessing

[SMP]

Definition From Wiki

Symmetric multiprocessing, or SMP, is a multiprocessor computer architecture where two or more identical processors are connected to a single shared main memory. Most common multiprocessor systems today use an SMP architecture. In case of multi-core processors, the SMP architecture applies to the cores, treating them as separate processors.

SMP systems allow any processor to work on any task no matter where the data for that task are located in memory; with proper operating system support, SMP systems can easily move tasks between processors to balance the workload efficiently.

Linux Kill All Processes By A Given Name

[Kill All Processes Under A Name, with grep, awk]


Suppose you want to kill all the processes containing name xyz, here is how you do it with grep and awk

kill -9 `ps u | grep xyz | grep -v grep | awk '{print $2}'`

very useful

Ubuntu Sony Vaio Laptop Turn off touchpad

[Turn Off Touchpad]

sometimes, the touchpad is annoyingly too sensitive, I will just turn it off so that it will not interfere my work, here is how I do it.

synclient TouchpadOff=1

Linux Sed Introduction

[sed]

sed stands for stream editor, I use it a lot to replace text patterns on the output stream.

for example, you have file named url contains a list of url's, when you use the cat it

cat url

it will show

http://www.example.org/a.html

http://www.example.org/b.html

http://www.example.org/c.html

http://www.example.org/d.html

http://www.example.org/e.html

run it with sed, you can do

cat url | sed 's/.html/.pdf/g'

then it will show the output as

http://www.example.org/a.pdf

http://www.example.org/b.pdf

http://www.example.org/c.pdf

http://www.example.org/d.pdf

http://www.example.org/e.pdf

very useful sometimes

Linux Locate Binary Program of a Command

[whereis command]

Ever wonder where you can find out where exactly is the binary for the "ls" command?

you can find out where the binary program of a given command by whereis

whereis ls

will show you where the binary program of "ls" is located

Linux Overview of Linux Kernel

Linux kernel provides a environment for programs and applications to run, I would like to think of it a resource manager that provides interfaces for the programs/applications to access hardware, such as cpu, memory, and network I/O.

Ubuntu Vertically Rotate Your Screen

You have a widescreen monitor? I prefer viewing my screen vertically when I am writing code.

to rotate your screen use the following command xrandr

xrandr -o left

or

xrandr -o right

one advantage of viewing stuff vertically is to reduce scrolling.

Linux Search Files By Name with find

[Find Command]

To find all c++ files in the sub directory use this

find . -name *.cpp

Ubuntu Setup LAMP Linux Apache MySQL PHP

Setup PHP5 + Apache2

apt-get install apache2 php5 libapache2-mod-php5

Setup MySQL

apt-get install mysql-server mysql-client php5-mysql

For L, you are already using Ubuntu Linux :)

Ubuntu Monitoring Network I/O

we can use watch command with netstat -i inet


so it goes like this

watch --interval 1 netstat -i inet

it will display the current network i/o status every second

Ubuntu Monitoring Disk I/O

[iostat command]

use the iostat command to monitor disk i/o every second

watch --interval 1 iostat

if you don't have iostat installed, run the following apt-get command to install it

sudo apt-get install sysstat

Linux Monitoring File System Usage

[df command to monitor file systems]

the one I like to use is

df -h

this will show the file system usage in "human readable" form

Ubuntu Monitoring Processes CPU Usage

[Use top command to actively monitoring your process]

top

top -d 10

change update delay to 10 seconds, default is 3 seconds

top -n 5

Refresh the screen 5 times, default is continuously refreshing

Linux Diff checking differences between files

[Use diff for checking file differences]

diff file1 file2

Linux iptables Block Incoming IP Address

[Block IP Address from 10.10.10.10 Accessing]

iptables -I INPUT -s 10.10.10.10 -j DROP

or

iptables -I INPUT -s 10.10.10.10 -j REJECT

[Remove The Block]

iptables -D INPUT -s 10.10.10.10 -j DROP

or

iptables -D INPUT -s 10.10.10.10 -j REJECT