Cali: road to alpha #1

When you need to invent a new programming language, you need clear and bold answer to simple question: «why».
Why you need new language at all. Why this is somehow new. Why there’s not a single other language which can satisfy you. Why this is not simple «nih».

These questions not just aggressive attacks, they even useful and good ones. If you want to invent new language, then first thing you should do is to find your own answers.

Let’s elaborate a bit about how programs are developed in world.

Currently, there are some SRS in some free-form doc, and according to them software are developed. Except few thoughts in light head of engineer, there are no connection between document and program. Changes made in SRS not magically appearing in program, and vice versa, some added code to program not transferring into requirements in document.

To fill this gap, there appeared several of professions — such as QA officers, testers, tech writers/reviewers etc. But this is not very flexible approach, since it’s still loose connection.

Idea 1: Requirements must be part of project in Cali.

That’s it. Before you have a lot of code, doing something, you need to made a prototype, or SRS, or just place accents and stresses on desired behavior. I’m totally believe that this is part of project, of same degree of needfulness as some source file, or build script. Requirements, definitions, proposals all of them must be defined in clear and strict form. Why strict form? Because they all could be used not only by humans, but rather by programs.

Now time to make one bold move and declare every elements in Cali as Concept. Concept is abstraction form of knowledge, this can be class, object, file, requirement, set of classes, feature, logic or algorithm. Sounds crazy, I know. But I think it’s only way to evolve programming, further and above from current code-monkey culture.

Idea 2: Basic unit of Cali is abstract thing, codename ‘Concept’

What in common do have Concept «Class» and «Logic»? «File» and «provided method»? I think there’s no common base for Concept (even name).

Idea 3: ‘Concept’ have no properties (except unique ID, which is must exist to distinguish one Concept from another)

Read soon — which very basic Concepts will be part of language.

Rrdtool strange rounding produces seesaw graphs

 

I have need of monitoring some processes — for example, count of requests per minute to external services.

First choice to have monitoring and graphs tool was rrdtool. It pretty simple choice, since there’s not much alternatives around.

So, background: Software X makes requests to somewhere, and increments some local counter. Each minute it tries to dump value to rrdtool.

«Tries» is the keyword, since dump happens not by cron, but in non-priority thread instead.

Rrd db file created with such settings:

rrdtool create -s 60 testing.rrd DS:cnt:GAUGE:60:U:U RRA:AVERAGE:0.5:1:1440 RRA:AVERAGE:0.5:60:720 RRA:AVERAGE:0.5:1440:365

This all came from internet/mans/tutorials. Basically, I’m interested in each minute’s value of requests (so, I chosed GAUGE as advices rrdtool homepage).

Second part — data dump, it happens not quite each minute exactly, but instead around each 60000 msecs, maybe plus some time to call «rrdupdate». So, probably this happens each 60500-61000 millisecs.

Seems reasonable, isn’t it? For hour I’ll get maybe 59 data points instead 60.

But instead I got some strange rounded values in rrd database…

To show problem, I created empty file with specified above command, and run for a while such commands in cycle:

rrdtool update testing.rrd N:5

sleep 61

This is quite similar behavior that I have in my software. In each minute I have cnt value = 5. Due to update going not each 60 seconds exactly, I can accept that in base I’ll have some value which close to 5.0 but a bit less than it.

What I got instead you can see in this graph:

rrd_seesawThis is very bad graph for me. I can deal with 59 records with 5.0 and one missing; Or I can live with 60 values with 4.91, But I really need to see trends and dynamics…

Funny thing is, that GAUGE should be used (as recommend site) when you measure some parameter, like temperature, but it failing to deal with unregular updates.

I could use ABSOLUTE, with hacks: I need to multiply value from rrd by 60 when I’m plotting graph. Also, If I’ll change period of updates from 60 secs to some other value, I’ll get garbage set of data, which not represents anything useful.

The question is, are there any way to measure data, with getting close to actual data in rrd, using «N:» update, or this problem should be solved with hacks-bikes in every usage?

Or am I missing something? I’ll appreciate any tips, comments.

 

P.S. I recreated data file with bigger heartbeat = 1800, here’s graph:

testing.rrd-avg-hb1800

P.P.S. Culprit happens to be old forgotten script, which ticked every minute and just placed «0» to all data files in given directory!

Bind99 DNS amplification attack workaround

If you happen to got DNS spoof attack, there’s easy way to prevent it.

Maybe it will not work in your case, but it is working  just fine in my case.

1. So, what is exactly «DNS amplification attack» — you can read in google, but in simple words — your DNS server is used to flood some IP. Since you know which IP targeted (it’s specified as spoofed request source), you can simply block it. Yes, including legitimate traffic, but this is not bad as it seems.

2. How can you prevent it — simple answer almost no, you can’t. Yes, you can limit rate of requests, you can slip answers, but it’s half measures.

3. Okay, prerequisites then (named, ipfw, cron):

  • Make sure that you have some rule in ipfw lower than 10000 which will allow all your legitimate traffic!
  • enable logging in named.conf, at least for queries level (If you change settings which influenced on count and ordering of columns, make sure you’ll tweak parser itself, e.g. change $5 — to something which will work for you):

    logging {
    category «queries» { «debug»; };
    channel «debug» {
    file «/var/log/named.log» versions 3 size 10m;
    print-time yes;
    print-category yes;
    };
    };

    You don’t need big log file for this purpose, 10m seems reasonable.

  • create parser script which will gather any new IP from named.log (I think all queries mentioned «IN ANY» should be blocked, maybe draconian, but working.)

    #!/bin/sh

    pre=»/root/»
    /usr/bin/grep «IN ANY» /var/named/var/log/named.log* | /usr/bin/grep -v «127.0.0.1» > ${pre}fullLog
    /usr/bin/awk ‘{print $5}’ ${pre}fullLog | /usr/bin/awk -F»#» ‘ {print $1} ‘ | /usr/bin/sort | /usr/bin/uniq > ${pre}newIps
    /bin/cat ${pre}allIps >> ${pre}newIps
    /bin/cat ${pre}newIps | /usr/bin/sort | /usr/bin/uniq > ${pre}allIps
    /sbin/ipfw delete 10000
    for i in `/bin/cat ${pre}allIps` ; do
    /sbin/ipfw add 10000 deny all from $i to me
    done

  • put in cron your script, every few minutes will do.
  • eventually fully clear your badlist — add to cron something like «Every hour(day,month) clear /root/allIps»

 

That’s it, you kinda made this attack useless.

 

New programming language — CALI

Today, I’m announcing creating of new programming language, even new paradigm.

This probably are not revolution in programming world, but anyway, I see many reasons to create new language.

`Concepts And Logic Application` — or CALI, is the new-approach, fresh gulp of air in world of monkey-ware and copy-paste-source-code.

Let me explain why it is so.

In high levels languages, like Java there is massive usage of OOP, and this is … good? bad? Well, besides of personal predilections, it is just as is. You can define classes, extend it, make interfaces, inheritance, whatever. But, there is now no concept-reusage, only code-reusage. In low level langauges, there is small bits of logic behind code; which could be inlined, or re-used, but all those methods are loosely coupled.

The main Idea behind CALI is that everything is logic, or concept. Concepts could be viewed as set of described ideas or concepts.

For example in CALI could be defined Concept «Singleton», or «Factory». You can work with those concepts, extend it, modify it, change products, whatever. And this is not about editing some real end-code.

So, probably you get idea, CALI is about how programmer defines tasks and logics in his head, but not really how they render to code. Concept could be more than one Class, for example some concept will include a set of Classes with special characteristics.

First stage of this language is to get 0.1 Version, which will include Concept-editor, and one back-end renderer (this will be Java).

How is this different from meta-programming? That simple one, even in meta programming languages highest unit of language is only one Class, but not set of them.

The development will go at GitHub, source code will be published at version 0.1 stage.

My tech blog in my horrible english