Jucetice Forum

Using the dispatcher with the appropriate opcodes instead of direct calls to effEditOpen etc is not a bug, in fact it is the method which most plugins now use.  If you write a plugin you need to be aware that the host may do this.
If you introduce your changes, it will stop my plugins from working because although they create their own display connection they need to know the hosts Window ID in order that they can be reparented by the host.  This was / is provided quite 'legally' according to the specification by casting ptr as an X Window ID  (int)  - ptr is "system dependent window pointer" - You are proposing to encapsulate both the Window ID and the display* into their own struct.  I realise this is a trivial change but it WILL break existing plugins.  This will most likely break any compatibility with other existing closed source plugins as well so you will then have JOST supporting only its own version of the linux version of VST.

Drumfix wrote:

Please try this patch

diff -Naur jost-v0.5.4-old/src/ui/windows/VstPluginWindow.cpp jost-v0.5.4-new/src/ui/windows/VstPluginWindow.cpp
--- jost-v0.5.4-old/src/ui/windows/VstPluginWindow.cpp    2008-12-02 23:15:00.000000000 +0100
+++ jost-v0.5.4-new/src/ui/windows/VstPluginWindow.cpp    2009-12-25 23:18:47.000000000 +0100
@@ -273,6 +273,16 @@
                                     int mouseY)
{
}
+
+void VstPluginWindow::paint (Graphics& g) {
+
+    if (plugin != 0 && externalEditor != 0)
+    {
+        externalEditor->paint (g);
+    }
+
+    DialogWindow::paint(g);
+}

//==============================================================================
void VstPluginWindow::broughtToFront ()
diff -Naur jost-v0.5.4-old/src/ui/windows/VstPluginWindow.h jost-v0.5.4-new/src/ui/windows/VstPluginWindow.h
--- jost-v0.5.4-old/src/ui/windows/VstPluginWindow.h    2008-12-02 23:15:00.000000000 +0100
+++ jost-v0.5.4-new/src/ui/windows/VstPluginWindow.h    2009-12-25 22:09:40.000000000 +0100
@@ -110,6 +110,8 @@
     void minimisationStateChanged (bool isNowMinimised);
     /** @internal */
     void filesDropped (const StringArray& filenames, int mouseX, int mouseY);
+    /** @internal */
+    void paint (Graphics& g);
     /** @internal */
     void resized ();
     /** @internal */

looks like this have been applied:
http://code.google.com/p/juced/source/detail?r=163
but no difference. still the same behaviour..
- ccernn

Drumfix wrote:

axWindowImpl::axWindowImp {

  ...
        else if (aWinFlags&AX_EMBEDDED)
        {
               ...
          mHandle = XCreateWindow(
            gDP,
            XDefaultRootWindow(gDP),    <--- here you must supply the parent window that as been given to you in the effEditOpen call
            aRect.x,aRect.y,aRect.w,aRect.h,
            CopyFromParent,
            CopyFromParent,
            InputOutput,
            CopyFromParent,
            CWBackPixmap|CWEventMask,
            &swa
          );
    ...

thanx for the help!!!
just tried this, and now, the code path goes something like this:

in axPluginVst.h:


case effEditOpen:
  if (mFlags&pfl_HasEditor)
  {
    #ifdef linux
      if (gDP==NULL) gDP = (Display*)value;
    #endif
    openEditor(ptr); // see below
    result = 1;
    mEditorIsOpen = true;
  }
  break;
  
void openEditor(void* ptr)
  {
    axWindow* win = (axWindow*)doCreateEditor((int)ptr); // see below
    if (win)
    {
      win->reparent((int)ptr); // see below
      win->show();
      mWindow = win;
    }
  }

virtual void reparent(int aParent)
  {
    mParent = aParent;
    XReparentWindow(gDP,mHandle,aParent,0,0);
  }


in axSynth.h (& axEffect.h):


virtual void* doCreateEditor(int aParent)
  {
    axEditor* ED;
    ED = new axEditor(WIN_NAME,this,-1,axRect(0,0,mWidth,mHeight),AX_FLAGS,aParent);
    do_SetupEditor(ED);
    mEditor = ED;
    return mEditor;
  }


in axWindowX11.h:


axWindowImpl(axString aWinName, axWidgetListener* aListener, int aID, axRect aRect, int aWinFlags, int aParent)
: axWindowBase(aWinName,aListener,aID,aRect,aWinFlags)
  {
    ...
    mHandle = XCreateWindow(
      gDP,
      //XDefaultRootWindow(gDP),
      (Window)aParent,
      aRect.x,aRect.y,aRect.w,aRect.h,
      CopyFromParent,
      ...

but, unfortunately, no luck... no difference...

Drumfix wrote:

Juce checks, that there is a child window of this parent after the return from effEditOpen,
and if there is not, it will ignore your _eventproc. This is what happens.

i draw things only as a response to Expose events (or WM_PAINT on windows), and the editor is drawn/shown correctly the first time it's opened.. wouldn't that indicate that jost know about my eventproc and sends me Expose events initially?? or else the editor would never be drawn??

also, i call XReparentWindow. won't that make my window a child of the parent window?

also #2, after switching back and forth between parameters/interface tab, jost send a Display* set to 0 (null). don't know if that could indicate something? (i currently 'cache' this in gDP, if gDP is null, and use that everywhere)

what should i try next? :-/

another screenshot.. i like screenshots... (with some debug variables/printouts)



- ccernn

update: a quick video to show how it looks in energy xt 2, qtrctor, renoise and jost.
ignore the other weird bugs, it's not a finished thing, it's for experimenting and testing hosts
(and forget the reaper part in the end, forgot that i hadn't compiled this test-plug to win32)

http://dl.dropbox.com/u/249632/jost/plug_debug.ogv

Last edited by cern.th.skei (2010-03-14 16:57:22)

Drumfix wrote:

Sorry, didn't see that you perform the reparent one level up.

Can you check, whether your eventhandler is called everytime you expect it to be called ?

added in some printouts in all event handler cases...

clicking or releasing the mouse causes correct (it seems) ButtonPress/Release events,

there are Expose events coming at various times, like after i mouse click, when entering the window with the muse cursor, etc... places where i don't expect new painting to be needed..

when i need to redraw the widgets, i call this:

virtual void invalidate(int aX, int aY, int aW, int aH)
  {
    static XExposeEvent ev;
    ev.type     = Expose;
    ev.display  = gDP;
    ev.window   = mHandle;
    ev.x        = aX;
    ev.y        = aY;
    ev.width    = aW;
    ev.height   = aH;
    ev.count    = 0;
    //TODO: test true
    XSendEvent(gDP,mHandle,false,ExposureMask,(XEvent*)&ev);
    //eventhandler( (XEvent*)&ev);
    //XFlush(gDP);
    //XClearArea(gDP, mHandle, aRect.x1, aRect.y1, aRect.width(), aRect.height(), true);
    //XSync(gDP,false);
  }

and then catch the Expose event in the event handler..
(all the commented stuff is from when i was debugging/testing)

is there any caching going on? anywhere i really should add in XFlush of XSync ??

another video (a picture says a thousand words they say, so a video would be millions.. :-)

http://dl.dropbox.com/u/249632/jost/events.ogv

a thing i just noticed, after i switched back from parameters view, i get no more events at all, and when exiting jost, there's a crash!

- ccernn

and, thanx for having patience with me!

kRAkEn/gORe wrote:

do you have a message thread running or do you expect that jost will call your event handler ? how do you tell your widgets to repaint themselves when you drag the mouse ?

have you tried also using XClearArea instead of sending an Expose message directly ? just guessing. the windowing stuff was a little bit tricky to write, probably we have to start from scratch again with simpler X11 calls and test your plugs with that (so we can work out what is the best way to do it in jost, which i think is far from being perfect in this case).

for standalone executables i have a message loop (on my todo list is experimenting with this in a separate thread):

virtual void eventLoop(void)
  {
    XEvent ev;
    while (1)
    {
      XNextEvent(gDP,&ev);
      ...
      eventHandler(&ev); // below
    }
  }

for plugins i rely on the host (or x-server or whatever?) calling my eventProc (using XChangeProperty, and '_XEventProc' and '_this'). took the basic plugin code from jorgen aase's synth4 example, which btw seems to have the same problem.

void eventProc(XEvent* ev)
{
  axWindowImpl* win = (axWindowImpl*)getProperty(ev->xany.window, XInternAtom(gDP,"_this",false));
  if (win==0) return;
  win->eventHandler(ev);
}

both calling into the real eventHandler(ev) in the axWindowX11 class

virtual void eventHandler(XEvent* ev)
  {
    ...
    switch (ev->type)
    {
      case Expose:
      ...
...

when widgets need to be redrawn, i (after some string of events) finally call the invalidate method (in my previous post), which should send an 'Expose' event directly to my window/event handler? (the above eventProc), and then i do the painting when i later receive these Expose events.

i do some additional duplicate widget checking, and update-rect merging in later sub-classes (axEditor, ..)

i have tried to do absolutely all painting in the expose event handler, because i have been afraid of creating threading issues later (it's all completely lock-free at the moment), and axonlib is for both win32 and linux, and i tried to 'merge' the two code-paths as low level as possible.

i think i have experimented with XClearArea a while ago (because i see it commented out in the invalidate method), but don't remember why i did it the way it is.. i've mainly used ext2 when debugging the code, and was testing the plugins in jost (and noticing the problems) quite late in the process.. but i can play around a bit more with it, and see if anything happens..

if it would help, i can make example/test/debug plugins quite quickly, with lots of debug-text output, etc. just say what would be most helpful...

- ccernn

update: yet another video, showing some differences between how the events are received from energy xt2 and jost
http://dl.dropbox.com/u/249632/jost/jost3.ogv (4mb, 2min, ogg video)
sorry about the mail message in the middle there :-)

Last edited by cern.th.skei (2010-03-15 00:37:01)

first of all, i need to say that i'm far, far from knowledgeable in how the xlib/x-server really operates 'under the surface', so most of what i know comes from guessing, experimenting, and crawling the web for hours, days, months for info and blog posts, and source code, and whatnot... i could be wrong in many areas, and would be happy to learn more about the internals, and 'getting it right'. so, correct me where i'm wrong! i'm always interesting in learning more! xlib doesn't really have very good docs/manuals available (or i haven't found them)

kRAkEn/gORe wrote:

I see the problem, Jost clearly don't know or intercept when the plugin window wants to be redraw, so it doesn't issue an Expose event to it.

your post got me thinking about my invalidate() method. i previously set the destination window, that needed to be updated to my own window.. that probably bypassed jost, and sent the event directly to my handler... tried to change that to mParent (the window i get from effEditOpen),

and, kaboom! it works!!!
at least the Expose events!
switching back and forth between interface/parameters still makes jost stop sending _any_ event to the window... but we're getting closer!

kRAkEn/gORe wrote:

Anyway i think the best way is to have an own event loop for each plugin window instead of relying on the host to call the correct dispatch event on our own window: this way the plugin can control itself quite completely and process any kind of message (have you tried adding ClientMessage in your eventProc window to intercept drag and drop messages to that window for example ? even EnergyXT2 cannot send those, as it will only send mainly mouse, keyboard and expose events).

why do my plugins work correctly in qtractor, energy xt2 and renoise?
what do they do differently from jost regarding those events?
(haven't looked too closely into juce(d) or the qtractor sources. sounds like a major task to learn the internal details of these large source bases)

and about ClientMessage stuff:
i use this for standalone executables, but mainly to get rid of some x errors when closing the window:

  wmDeleteMessage = XInternAtom(gDP,"WM_DELETE_WINDOW",True);
  XSetWMProtocols(gDP,mHandle,&wmDeleteMessage,1);
  + ClientMessage bit set in eventmask when creating window
 
and my event loop receives these:

  virtual void eventLoop(void)
  {
    XEvent ev;
    while (1)
    {
      XNextEvent(gDP,&ev);
      unsigned int data = ev.xclient.data.l[0];
      if (ev.type==ClientMessage && data==wmDeleteMessage) break;
      else eventHandler(&ev);
    }
  }
(i just ignore that event here).
haven't used that in a plugin window, as there's no 'X' button or close window stuff to bother with there. (and perhaps i should use mParent instead of mHandle up there too... need to test this a bit more..)

but, would i receive different events from XNextEvent than i would receive in eventProc?

kRAkEn/gORe wrote:

I'm not a fan of that eventProc procedure, and that sounds to me like a hack in itself. On windows for example, in which you have a message loop running on every window (intercepted by the windowproc), the host don't call anything on the plugin directly, but instead post messages to its window, and it's the plugin message loop that does all the logic (in windows is done by the system, on linux you should need a thread to process that). Passing a function to the host isn't a safe method here, what if i want to intercept a new message in my plugin window ? i need to wait that the host is rewrote to pass me the new message which it may not know anything about ?

i kind of like the eventProc method, makes the code/logic in linux version and windows version behave quite similar, gives me one single entry/exit point for events, without adding additional complexities like threading, locking, etc...

why not just send the event(s) unaltered to the plugin window? and let the plugins do what they want with them? or, perhaps, just the events you don't handle yourself in jost?

kRAkEn/gORe wrote:

So here i think we have to make some changes to the specs for linux, requiring the vst plugins windows to have their own message thread. Juced plugins don't have this problem cause they have their own message thread running, and don't pass any info to the host.

i personally don't like the sound of required..
because, well,  ext2, qtractor, renoise, etc, works quite fine with eventProc, so it must be possible for the host to handle it correctly. i would rather see the plugin developer having the choice. if anybody needs more advanced event handling, they need to use threading and event polling/reading.. but if not.. their choice.


and, last:

i see that hypercyclic have some problems too, when switching back and forth between interface/parameters tabs... it's not redrawn correctly after switching back to the gui, and i can't tweak the knobs anymore...

i also get some error (and it hangs, i need to kill the process manually) when exiting jost:

*** glibc detected *** ./jost: double free or corruption (out): 0xb6e0b678 ***
ERROR: Program executed illegal instruction.. terminating

that could be my code, though... debugging, here i come :-)

- ccernn

cern.th.skei wrote:

first of all, i need to say that i'm far, far from knowledgeable in how the xlib/x-server really operates 'under the surface', so most of what i know comes from guessing, experimenting, and crawling the web for hours, days, months for info and blog posts, and source code, and whatnot... i could be wrong in many areas, and would be happy to learn more about the internals, and 'getting it right'. so, correct me where i'm wrong! i'm always interesting in learning more! xlib doesn't really have very good docs/manuals available (or i haven't found them)

your post got me thinking about my invalidate() method. i previously set the destination window, that needed to be updated to my own window.. that probably bypassed jost, and sent the event directly to my handler... tried to change that to mParent (the window i get from effEditOpen),

and, kaboom! it works!!!
at least the Expose events!
switching back and forth between interface/parameters still makes jost stop sending _any_ event to the window... but we're getting closer!

This is a problem i need to figure out from the beginning, ie not using juce, but plain X11 calls. After i got something working with plain X11, we can start plugging it inside the juce code.

cern.th.skei wrote:

why do my plugins work correctly in qtractor, energy xt2 and renoise?
what do they do differently from jost regarding those events?
(haven't looked too closely into juce(d) or the qtractor sources. sounds like a major task to learn the internal details of these large source bases)

what we do differently is that we manage the window in the juce fashion, so create a juce window and add it to the desktop, but we don't manage to work with x11 calls directly (which may lead to a better handling). Renoise for example, create a normal x11 window without applying its framework over it (as you may see).

cern.th.skei wrote:

and about ClientMessage stuff:
i use this for standalone executables, but mainly to get rid of some x errors when closing the window:

  wmDeleteMessage = XInternAtom(gDP,"WM_DELETE_WINDOW",True);
  XSetWMProtocols(gDP,mHandle,&wmDeleteMessage,1);
  + ClientMessage bit set in eventmask when creating window
 
and my event loop receives these:

  virtual void eventLoop(void)
  {
    XEvent ev;
    while (1)
    {
      XNextEvent(gDP,&ev);
      unsigned int data = ev.xclient.data.l[0];
      if (ev.type==ClientMessage && data==wmDeleteMessage) break;
      else eventHandler(&ev);
    }
  }
(i just ignore that event here).
haven't used that in a plugin window, as there's no 'X' button or close window stuff to bother with there. (and perhaps i should use mParent instead of mHandle up there too... need to test this a bit more..)

but, would i receive different events from XNextEvent than i would receive in eventProc?

this is ok as you have your message loop that will catch the event (waiting in the XNextEvent). but for plugins that don't have the XNextEvent call, we rely on the host, and the host may or may not forward those messages to the plugin window (which is bad imho).

cern.th.skei wrote:

i kind of like the eventProc method, makes the code/logic in linux version and windows version behave quite similar, gives me one single entry/exit point for events, without adding additional complexities like threading, locking, etc...

why not just send the event(s) unaltered to the plugin window? and let the plugins do what they want with them? or, perhaps, just the events you don't handle yourself in jost?

well, it is more similar to how code is written, but not how things are working. in windows, it's not the host that forward events to the plugin itself, instead is the window internal event loop.
sending unaltered messages to the window requires to rewrite most of the jost handling of the plugin window with reparenting.

cern.th.skei wrote:

i personally don't like the sound of required..
because, well,  ext2, qtractor, renoise, etc, works quite fine with eventProc, so it must be possible for the host to handle it correctly. i would rather see the plugin developer having the choice. if anybody needs more advanced event handling, they need to use threading and event polling/reading.. but if not.. their choice.

ok, this can be developer selectable, but i find the event proc method a quick hack to get things done. not a proper way of handling.

cern.th.skei wrote:

i see that hypercyclic have some problems too, when switching back and forth between interface/parameters tabs... it's not redrawn correctly after switching back to the gui, and i can't tweak the knobs anymore...

i also get some error (and it hangs, i need to kill the process manually) when exiting jost:

*** glibc detected *** ./jost: double free or corruption (out): 0xb6e0b678 ***
ERROR: Program executed illegal instruction.. terminating

that could be my code, though... debugging, here i come :-)

- ccernn

as i said, this side of jost need to be carefully rethink from the ground up...

Drumfix wrote:

The "hack" doesn't work on 64 bit anyway.

why not?
because the value argument is 32-bit?
just had a look at the vst sdk (v2.4), and see this in aeffect.h:

typedef int32_t VstInt32; ///< 32 bit integer type
typedef int64_t VstInt64; ///< 64 bit integer type

#if VST_64BIT_PLATFORM
typedef VstInt64 VstIntPtr; ///< platform-dependent integer type, same size as pointer
#else
typedef VstInt32 VstIntPtr; ///< platform-dependent integer type, same size as pointer
#endif

and the dispatcher (where we receive the Display* from the 'value' parameter) is:

virtual VstIntPtr dispatcher (VstInt32 opcode, VstInt32 index,
  VstIntPtr value, void* ptr, float opt);

to me, that looks like it's already 64-bit safe?
or have i overlooked/forgotten something?

Drumfix wrote:

@ccernn: If you use your own connection to the X server in your own thread, you receive all events you specified in the eventmask supplied to XCreateWindow.

ok.
so, if i use the display from the effEditOpen, i get only the events the host are sending to me?
tried a quick experiment with my own thread, but forgot about this display connection, and only got hangs/freezes :-/

Drumfix wrote:

As i wrote earlier in this thread, the correct way would be to send a char* for the display name + the XID of the window to use as parent in the effEditOpen call and
then let the plugins open and maintain its own connection to the x server.

what about using some of the VendorSpecific things in the vst sdk? reaper uses that for some of its extensions...
read 'vendor' like 'platform', and it would make sense :-)

- ccernn

i think we are over engineering things here. and adding too much specific bloat to linux handling of vsts.

from my point of view, new plugins should not use eventProc anymore, which was a sort of hacky "extension" introduced by EnergyXT2 developer to make it's own code work.

a little progress!
a message thread up and running, i think :-/
(and i finally managed to send a custom ClientMessage to stop the thread)
i can tweak the knobs, switch between interface/parameters tab, close, reopen, etc...
but the gfx is drawn in the wrong place, over the juce window controls & title bar...
(maybe related to the double Expose messages i mentioned earlier?)
mouse stuff is kind of working, but wrong..
as if the mouse coordinates are related to my own editor window,
while the gfx coordinates are related to the full (including title bar, etc) juce window...



and:

- renoise: completely hangs when trying to open the editor, needs manual process killing
- qtractor: just a blank (white) window, nothing else, i can open/close the editor
- energy xt2: nothing drawn at all, and x crashes (BadWindow/X_DestroyWindow) when closing the editor, and trying to open it a second time)

so... things are not looking very promising at the moment.

luckily i've encapsulated all the new threading stuff in
#ifdef AX_THREAD_GUI ... #endif
so that i can still make plugins that work everywhere else

- ccernn