IRC Devel Meeting - 2012-01-31

Date:

  • 15:00 UTC, Tuesday, January 31, 2012

Utilities:

Place:

  • #sip-router IRC channel on irc.freenode.net server

Agenda

Kamailio related:

  • (dcm) outstanding issues at this time, if any
  • (dcm) packaging - ways to automate the process better
  • (dcm) documentation - suggestions to improve it and attract more contributions in this field
  • (dcm) 'librarization' of presence modules - is there duplicated code that fit in an internal lib?
  • (dcm) merging duplicated K|S modules
  • (dcm) unique ids for user profiles
  • (JP & RG) IMS branch and related modules
  • (dcm) kamailio 3.3.0 - roadmap to next major release
  • (dcm) ideas for new features
  • (dcm) administration - web site, mailing lists, etc ... do we miss some good resource out there that should be added in the eco-system?

Participants

Following developers and community members announced intention to participate (add yourself, alphabetic order, especially if you add items in the agenda):

  • Daniel-Constantin Mierla (dcm)
  • Elena-Ramona Modroiu (erm)
  • Jason Penton
  • Richard Good

Note that participation is open to anyone, just join the IRC channel if you want to participate.

Transcripts of Discussed Topics

This is pretty much the unedited IRC discussion, split by topics.

Existing Issues

miconda: anyone here to report any outstanding issue (code, administration, etc.) that needs to be fixed?
richardgood: nothing to report here.
miconda: ok if none is quick with something ... think about and eventually say during this session

Packaging

miconda: second: packaging
miconda: we have debs and rpms
miconda: debs provided at deb.kamailio.org are fine for nightly builds
jasonpenton: maybe an option for Solaris packages - but I'm not too sure of the demand?
miconda: but the release builds need some manual triggering, as I understood from Jon Bonilla
miconda: and they stay sometimes long time behind
miconda: for that reason I made an OpenSUSE Build Service that can be triggered via web
miconda: I will post some details later
osas: is there a real need for nightly pkg builds?
miconda: rpms should be fine for now on OBS (OpenSUSE Build Service) - rpms for centos, redhat, fedora and opensuse are generated
miconda: osas: yes, they are very useful to get fixes from stable branches
miconda: agranig: thanks, but I was looking for a solution when many people can get access
osas: besides for checking that the build is susscessfull, I don't see a real presure in having nightly builds for all platforms
miconda: not for all platforms
miconda: but actually the problem is not nightly builds
miconda: they are done automatically
miconda: just release builds
agranig: miconda: sure... just as a side note: we're using jenkins to build our internal debs and generate release repos, maybe this should be considered for kamailio as well?
miconda: which have to be done manually, since the releases happen sporadically
agranig: i can provide some input on that if we want to go that path
miconda: jenkins is a service or tool?
agranig: it's a tool... a continuous integration server, which can be hooked up with a git repo, which than triggers deb builds either immediately upon commit, or on nightly intervals, or when you prepare a release
miconda: at this moment the debs are generated on a server that would require user access to trigger release builds
miconda: and it is hard to keep adding users to be sure someone is going to be available to trigger
miconda: OBS does everything over web
miconda: or a client installed on own system
miconda: so it is convenient to allow "any" people to manage the repository and revoke later if not trusted
agranig: ok... with jenkins, you'd have a web interface where you control it (and a jabber interface etc)... it's quite powerful in that regards... you'd still have to maintain users though
miconda: I am not saying is the best way, that;s why I wanted to see if there are other options
miconda: ok then, let's see if we can get something with jenkins or obs in regards to debs for releases ...
miconda: jasonpenton: solaris would be nice
miconda: if someone can take care of
miconda: I am not a solaris user at all
jasonpenton: look - we will be doing it for ourselves anyway - so we will take it on
miconda: great, thanks!
jasonpenton: but can't commit a time yet!
miconda: no problem!
agranig: there are couple of tools to maintain repos automatically, we can talk at fosdem about the details
miconda: *BSD specs are probably old as well ...
miconda: gentoo got quite up to date
miconda: agranig: ok
miconda: so, to finish this topic
miconda: if you know tools to automatize building packages for any distro or want to maintain packaging for some distro, write to sr-dev mailing list

Documentation

miconda: next topic: documentation ...
miconda: I saw lot of good info going though mailing lists, but nobody is digesting in some wiki pages or faqs ...
miconda: anyone has ideas of how to attract doc writers?!?!
osas: It should be the responsability of the one who get's help to update the wiki
miconda: yes, but cannot be enforced
osas: and the pone that provides help should remind the one that gets helped to update the wiki
osas: it cannot, but if someone is reminded to do so, it will (most of the time)
jasonpenton: users will never update/contribute to docs
miconda: so we need to engage their willingness somehow
jasonpenton: it will be up to devs only IMO
osas: no
osas: devs have a different POV
osas: I cannot see things like a beginner
osas: and therefor I cannot update docs from a beginner POV
osas: I think that's the issue
jasonpenton: true - but users are typically consumers of info and never interested in producing
osas: from my POV, the docs are quite good
osas: Than you get hellped onces, and that's it
miconda: ok, so kind of bonus program ...
jasonpenton: lol - that;s a good idea - provide doc for free consulting
miconda: you got answers and no updates to docs, then you are delayed
osas: I think a friendly reminder by everyone who provides help shold give the docs a boost
miconda: osas: yes, that should be done, added as a signature
miconda: jasonpenton: could be something like audio conference
agranig: just curious... where would you even add interesting junks from the mailing list, e.g. config snippets for specific use cases?
osas: and a friendly monthly e-mail about it
jasonpenton: agranig: wiki
jasonpenton: ?
agranig: yeah, but where?
miconda: aswering questions to participants and they have to write docs based on answers
jasonpenton: what about the responders to good email questions: create a wiki page and the user needs to update on experienceand succuess, etc?
miconda: maybe 1-2 sessions a month
agranig: like in http://www.kamailio.org/wiki/start#tutorials or where?
jasonpenton: and all follow up questions
miconda: jasonpenton: sometimes I answer from a mobile or in a hurry, creating a wiki is not always easy...
miconda: agraning: yes, that is a place, other can be created, like FAQ page
miconda: ok, other ideas? If not, we will digest this discussion and go to mailing list for follow up
agranig: handing off good questions to the wiki sounds interesting, but the usability is questionable... you don't get to know if/when the OP is providing infos there etc
miconda: agraning: first is to get something, then we can review and clean up
miconda: but how to get some writers...
agranig: maybe a Q/A portal similar to stackoverflow could help, but then again you could just stick with the mailinglist...
qxork: I was thinking about this hard based on the mail yesterday... problem is that without a very good understanding of SIP, the documentation simply won't make sense.
qxork: I think what Alex wrote yesterday was amazing
agranig: ack
agranig: it should be taken as kind of an official project introduction
miconda: qxork: access to web site only if answering correctly a SIP spec question?
qxork: I do think Alex's piece might make a very good intro

Presence

miconda: next a bit about presence ... I added recently based on last mailing list activity
miconda: is pdunkley or other presence users/devs active in the channel?
pdunkley: I am here
miconda: I got the impression that some code is duplicated across couple of modules
miconda: when presence server was started, we had no internal libraries
miconda: but now we have that
pdunkley: There is certainly quite a bit of code duplicated within some of the modules.
miconda: would it be the case of making an internal lib to collect such code?
pdunkley: The only stuff I can think of that is duplicated between the modules is the pres_auth_status()/xcap_auth_status() stuff.
miconda: so there will be no need to patch in many places, whenever is something to fix
miconda: btw, irc, presence module has also some lib only mode
pdunkley: I am not sure how easy it would be to write a lib for some of the stuff.  I don't think there is a huge amount duplicated between presence/presence_xml/pua/rls.
pdunkley: There is similar (but not quite the same code) in some places.  But not a lot duplicated.
pdunkley: I don't know about the other presence modules (for dialogs, pua_xmpp, and so on).
miconda: ok
miconda: wanted to double check ... when I have time I will look a bit over the code
pdunkley: I think there could be a lot duplicated between presence_conference, presence_dialoginfo, presence_mwi, ... though
miconda: ok
pdunkley: A nearly identical pair of files (pidf.[ch]) is in presence_xml, presence_conference, presence_dialoginfo, and pua.
pdunkley: and pua_xmpp
pdunkley: So there could be something there.
miconda: thanks for checking, maybe I have time to look over before Fosdem and update there for a decision
miconda: anything else related to presence?
miconda: any refactoring that should be done?
miconda: anyone using MSRP?
pdunkley: I am currently thinking about adding an internal API between RLS and presence.
pdunkley: I might clean-up some bits and pieces as I do that, if I spot anything.
pdunkley: Internal as in functions, to try and squeeze a bit more performance and dodge some of the race hazards.
miconda: ok ... topic done?
carstenbock: The topic of my speech at LinuxTag will be on Rich Communications with sip-router.org (if accepted)
pdunkley: I've not tried the MSRP yet.  I was thinking a handy feature might be some sort of function interface into MSILO so that offline MSRP messages can be captured?
miconda: offline msrp ... if the connection to target is closed, could be useful
pdunkley: Yes, and establish a connection and send the messages when the target REGISTERs and comes online.
miconda: pdunkley: but you have to create a dialog with INVITE first ...
pdunkley: Can the uac module (or other modules) be used to terminate the INVITE session and create an INVITE session after REGISTER?
miconda: uac does not have for the moment such feature
miconda: can be enhanced
miconda: dialog module has some feature for doing click-to-dial
aXl: regarding pua: i still have to port the deadlock freen version from 1.5 to 3.x. I looked at it quickly a few months ago, but it will require a bit of work for which i don't have time yet.
miconda: invite, 200ok, ack, refer, 200ok, bye, 200ok
miconda: aXl: i think that was fixed
miconda: was it deadlock or race in handling taffic?
aXl: it deadlocked because it kept a lock on a hash bucket entry while waiting for a sip response
aXl: if all sip listeners are waiting for such a lock...
miconda: I had in mind it was fixed by some refactoring in handling the traffic
miconda: you can doublecheck when you have the time
aXl: ok, i haven't noticed any siginficant changes to pua in the 3.0/3.1 timeframe, so i assumed it wasn;t fixed yet
miconda: there were lot in 3.2 and devel
pdunkley: DB only mode and a lot of race-hazards fixed since 3.2
miconda: anything else on presence?
pdunkley: Nothing from me
aXl: no

Duplicated Modules

miconda: merging duplicated modules
miconda: long live topic
miconda: I hope we get pdt, dispatcher and few more merged
miconda: auth_db depends on db schema, let's see
miconda: maybe usrloc, as I have some plans for gruu support
jasonpenton: miconda, quick question about modules K|S - if we write a new module, where should it go? K|S|modules?
miconda: ser version has implemented parts of gruu already
jasonpenton: assuming brand new functinality
miconda: should go in modules
jasonpenton: ok
miconda: radius modules were merged once, then reverted, don't recall the reason right now
miconda: xlog from ser still has specifiers like '%ru' and it is used by other ser modules
miconda: perhaps is gonna take more to get to a decision for it
miconda: speeddial, cpl-c and permissions might be also easy ...
miconda: just to give the current state
miconda: other modules from modules_k/s can be just copied, they are not duplicated, just have some local dependencies
miconda: if any dev has spare time to work on reducing duplicates, write on sr-dev to sync
aXl: it'll probably need a dev with experience in both projects, at least with the module to be converted, i don't think there are a lot of them
osas: there should be a wiki page dedicated to module merging
osas: and maybe it makes sense to just drop one version and keep the other one (for some modules)
miconda: when I did it, I looked in readmes to be sure the features are not lost
miconda: most of complexity comes from different db structure
osas: for some
osas: for some others, it should be relatively simple
osas: like nathelper
jasonpenton: I assume in a merge all we need to preserve is 1. functionality, 2. API, 3. DBSchema? is that all?
osas: there are some extra features in s version, but OI don't know how many are relly using those
miconda: jasonpenton: ideally
miconda: for db schema I try to make it compatible with both
miconda: maybe by having some columns used by only one "mode"
jasonpenton: miconda: implemented how? #ifdefs?
miconda: no, modparams
jasonpenton: ok
osas: for some modules, maybe we need a pool to see much are those used
miconda: I prefer no compile time switches
jasonpenton: yeah was gonna say
miconda: it is like with use_domain parameter
osas: and based on that, consider dropping on version and keeping the other one
miconda: domain column is not used if that parameter is not set
jasonpenton: ok
miconda: osas: yes, at some point we should get to something like that
miconda: related to this, I added also the next topic
miconda: adding kind of unique id in some structures
miconda: ser uses that and might be easy to integrate
miconda: we can make it as user@domain from existing records
miconda: the idea showed up when I looked a bit over gruu specs
jasonpenton: please explain a little miconda, im lost here
jasonpenton: you are talking about location db schema?
miconda: jasonpenton: at this moment we have (username) or (username,domain) to identify a subscriber
jasonpenton: ahh yes - okay
miconda: a matter of use_domain, there are conditions and conditions
jasonpenton: we had issues with that in IMS extensions too
jasonpenton: ok
miconda: so it might be good to have a main SIP id the (username,domain) and unique id
miconda: sip id can change eventually
miconda: so it is about adding a new column in subscriber and location tables
miconda: for auth and location services
miconda: not sure what will be the impact
jasonpenton: happy with that!
miconda: wanted to see if people see benefits
jasonpenton: def!
miconda: the unique id target is to be used internally for references between records
jasonpenton: well for IMS extensions, we have to have 'custom data' added to customer profile - and for backwards compat we had to use another DB table
jasonpenton: keying with username/domain is not cool
jasonpenton: uniqueid will be much  better
miconda: in gruu is so called permanent address, which has to be mapped to a subscriber
miconda: ok ... we have to see the impact anyhow, might not be for next release (or at least not across all modules)

IMS

miconda: then since you are on stage, the next topic IMS
miconda: I added it in case someone wants to ask questions and see the status
miconda: there are two branches, afaik
jasonpenton: yeah - not much Daniel - we have a working prototype whcih we are testing
miconda: carstenbock and jasonpenton
jasonpenton: however, one thing we are puzzled over is how to integrate leveraging existing functoinality
jasonpenton: mainly in usrloc/registrar
carstenbock: I actually point everyone to Jason's branch.
carstenbock: Since i think it is more advanced than my branch.
miconda: ok
jasonpenton: as I mentioned earlier 'custom data' for subscribers  - do you think it would be feasible to include in usrloc a 'custom field' to add anything - using some form of serialisation?
miconda: jasonpenton: usrloc from modules_s has that functionality
miconda: a reason i was looking to merge
jasonpenton: ahh - i didnt notice that
miconda: it uses avps
jasonpenton: I have the bad misconception that k is better than s
miconda: in many aspects
jasonpenton: ok will look into - we really just dont want to duplicate functoinality
miconda: k version has lot of features not in _s
miconda: I didn't add in _k/usrloc because I wanted to merge first (and therefore get it)
jasonpenton: ok - so our roadmap is to finish testing pretty soon and then push our stuff - so maybe ready for by the  ++next release
miconda: ok
jasonpenton: other than that no updates
jasonpenton: also hoping someone could look at dialog2 for us
miconda: to implement it?
jasonpenton: we're not experts in TM and dialog - but dialog 2 works for us
jasonpenton: no Richard did alot of the work already
miconda: is it on git?
jasonpenton: yes
jasonpenton: in our branch
miconda: i missed it then
miconda: ok
miconda: i will try to find time for it
jasonpenton: okay no rush
jasonpenton: so topic done
agranig: one thing regarding usrloc from my side is the sync with db... we've talked on the sr-users list a bit about that, where we should make the write-back a bit more resilient (insert vs update, independent from the state in cache)... I couldn't come up with some generic approach yet
carstenbock: Great
miconda: agranig: indeed, it was a topic
miconda: let's finish this one then
miconda: solutions?
miconda: someone proposed a new db mode, just for read only at startup
miconda: and all registers will be replicated to another instance to write to db
agranig: well, one could be to check in the update result whether rows where affected... this is just for mysql though, not sure about other modules
miconda: it might be a contribution soon, let's see
aXl: for mysql insert ... on duplicate key can be used it is faster than update, then insert (and racefree)
agranig: the problem we have atm is rather that the usrloc cache thinks it's already in db, and just tries an update (no insert at all)
aXl: above solves that, just use insert always
aXl: but only for mysql unfortunately
agranig: would this work with pgsql and oracle and other drivers we support?
miconda: aXl: or anyone,  is insert ... on duplicate good in postgres?
miconda: ok ... we have to check, I guess
nangel: no, I don't think postgres has insert ... on duplicate... I forget the way to do it though
 • nangel has mysql + postgresql dbs
miconda: ok, back to next topic ... if nothing else here
nangel: http://stackoverflow.com/questions/1109061/insert-on-duplicate-update-postgresql - create a function
aXl: if we create a new db op like insert_or_update
aXl: it can be solved within a transaction for every DB out there
agranig: or some stored procedure, yes
aXl: that requires DB magic, i'd prefer if de db module of kamailio would provide support for it
miconda: maybe a dedicated process to detect consitency
miconda: walk through mem records and check if in db ...
miconda: anyhow, not easy to decide here
miconda: let's continue on mailing lists

Roadmape to 3.3.0

miconda: road map to next major release 3.3.3
miconda: 3.3.0
miconda: so we released in October, we should do it before summer
miconda: nobody wants (I guess) to do it in holidays from the beach
miconda: maybe looking to freeze devel after mid of march, latest on end of march
miconda: test on april
miconda: release in may?
miconda: anyone having plans for major additions that require more time?
miconda: looks like no
miconda: so I will propose these time lines and see reaction on mailing lists
miconda: approaching the end of this session then
miconda: anything missing as feature in the project?
miconda: what you would like to see added soon?
miconda: (not saying it will be)
aXl: i have one
miconda: good, you have dev access
jasonpenton: lol
aXl: support for modifying the outgoing message _after_ DNS resolving
aXl: like i talked with you about on fosdem
aXl: there is a need to enable an rtpproxy for ipv6/ipv4 transistion situations
miconda: just after dns, before printing the output buffer
carstenbock: There was some hacks on the list recently, like using the local IP as outbound_proxy in some presence-modules...
aXl: it would also need to be able to update avp's or record-route parameters
carstenbock: It would be nice, if that could be avoided.
miconda: I will look for places where we can call an event route on forward, after dns, before building output
aXl: what would maybe work is onsend_route where this would be possibel
miconda: onsend_route has the outgoing buffer printed
aXl: just ned to prohibit rewriting $du
miconda: too late for route headers
aXl: then a new event would be needed
miconda: the tricky part will be with dns failover
miconda: ipv6, then fallback to ipv4
aXl: the event should be called for every outgoing message (apart from retransmissions)
miconda: ... then to noIP
aXl: that;'s exactly the issue im having
miconda: ok, understood
aXl: mixed ipv4 and ipv6 in one dns records
miconda: for records, tm module and forward.c are places to look at
miconda: anything else?
nangel: wishlist - would be handy to be able to do SRV/A/AAAA/NAPTR lookups directly from script
aXl: somewhere between creating the transaction and adding record-route headers ?
miconda: yes, Record-Route header is added after dns and selection of local socket to send
pdunkley: I have a suggestion for a new feature.  Can the rtpproxy module detect when the SDP is indicating TURN/ICE support in the client and only activate when it is needed?  Or possibly add something to SDPOPs so this can be detected in config and rtpproxy not used?
aXl: but when is the tranasction created? before or after that?
miconda: nangel: ok
miconda: before
miconda: but should work for stateless mode
miconda: with forward()
aXl: that would be a good thing i think
agranig: pdunkley: sounds like this will be much needed in the light of webrtc
miconda: in tm, the code of t_relay() should be followed and dug in
miconda: in core, forward() in forward.c
aXl: ok, thanks
pdunkley: agranig: I was thinking in the context of RCE/RCS-e too.  For those specifications NAT traversal in the client is mandatory - but some still don't do it.
miconda: pdunkley: sdpops can be a good place
pdunkley: So need to be able to cope efficiently with both types of client (good ones and bad ones :-))
miconda: I think it is about searching for a=ice and similar
pdunkley: It's been on my wish list to do this for a while, but I haven't had time.
miconda: ok
pdunkley: Yes I think is is something like looking for ice on an attribute line.
pdunkley: Maybe a generic SDPOPs to allow someone to match an "a=" line with a reg-ex would be useful for other things too?
miconda: search_body() does it
miconda: from textops
miconda: safe for sdp body only
pdunkley: I didn't think of that.
miconda: if it is a multi-part body, then can be conflicts a matter of content
miconda: so an sdp_search() would be safer
miconda: there is also a new transformation on devel
miconda: line: http://www.kamailio.org/wiki/cookbooks/devel/transformations#line_transformations
miconda: can be enhanced for matching on a line based
miconda: btw, IPv6 -- how is it in your side?
miconda: production already?
miconda: we should be pretty safe with everything related to IPv6
miconda: carrierroute and drouting were not checked by me so far
pdunkley: IPv6 is required for RCE/RCS-e as it is IMS based.  Lots of people still using IPv4 at the moment though.
miconda: from the list of routing modules
jasonpenton: that is somethign we still need to test too (IPv6)
agranig: our tests were pretty much fine with IPv6, but we don't use carrierroute and drouting...
pdunkley: So haven't done much with IPv6 yet myself.
jasonpenton: on our roadmap and we will be using cr
jasonpenton: as well as dispatcher
miconda: I don't use these modules as well, but from the list of modules I know the may deal with, are still in the check list
miconda: dispatcher should be ready
jasonpenton: great
miconda: ok then
agranig: at least the core, usrloc and lcr are definitely ready

Closing Remarks

miconda: last topic -- any relevant resources we are missing in the environment?
miconda: lot of cool apps might be out and some of us are not aware
miconda: facebook 'Love' page ;-)?
qxork: kamailio cake.
miconda: I know where to place the order
agranig: native websocket-transport-support proposed by inaki
agranig: i think he implemented it in his own server, not in kamailio
miconda: well, to my understanding so far to websockets (some sporadic reads), it will not be something very complex
miconda: it starts as an http
miconda: we have that
miconda: then there is a handshake based on some headers
miconda: then it is pretty much tcp
miconda: with packets to be handled based on app id or so ...
agranig:  what's missing is the WS(S) support in vias... you usually get a client id for a websocket, so you'd need to map a message coming via "normal" sip to the correct socket on the http-side
miconda: it is like msrp
miconda: you have to keep the mapping of connection id
miconda: msrp has no Via headers as well
miconda: ok ... added to the list
agranig: as far as i understood inakis proposal, websockets will have vias... it's intended to be another transport beside udp, tcp etc
agranig: anyways
miconda: not sure about Inaki's proposal
miconda: knowing a bit about generic websocket specs
miconda: so Inaki may have proposed some 'enhancements'
agranig:        http://tools.ietf.org/html/draft-ibc-sipcore-sip-websocket-01
miconda: anything else? otherwise looks we can close development in few days ...
jasonpenton: cool - ciao!
miconda: ok! then officially ending here!